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nnovation  has  become  a  very  popular  word  lately. 
Former  Secretary  of  Defense  Chuck  Hagel  an¬ 
nounced  the  Defense  Innovation  Initiative  about 
a  year  ago.  At  about  the  same  time,  the  draft  Bet¬ 
ter  Buying  Power  3.0  set  of  initiatives,  focusing  on 
technical  excellence  and  innovation,  were  published 
for  comment.  Deputy  Defense  Secretary  Robert  0. 
Work  has  led  the  effort  to  develop  an  innovative 
"Third  Offset  Strategy."  Most  recently,  Secretary  of 
Defense  Ashton  Carter  announced  the  opening  of 
the  Defense  Innovation  Unit— Experimental,  or  DIU- 
X,  in  California's  Silicon  Valley.  President  Obama  has 
led  the  administration's  successful  opening  of  several 
Manufacturing  Innovation  Institutes,  most  of  which 
are  sponsored  by  the  Department  of  Defense  (DoD). 
And  more  institutes  are  on  the  way. 

Today  it  is  possible  to  obtain  advanced  degrees  at  major  uni¬ 
versities  in  the  fields  of  innovation  and  entrepreneurship.  Many 
books  and  articles  have  been  written  on  innovation,  perhaps 
none  more  well-known  than  Clayton  Christianson's  "The  In¬ 
novators  Dilemma."  I  would  like  to  add  a  few  thoughts  to  that 
body  of  work  by  making  some  very  unscientific  (meaning  un¬ 
supported  by  data)  comments  on  the  ingredients  needed  to 
foster  and  encourage  innovation— and  on  the  extent  to  which 
the  DoD  acquisition  enterprise  has  or  does  not  have  those 
ingredients  today. 

The  first  and  absolutely  necessary  ingredient  is  knowledge. 
Technical  innovation  is  itself,  almost  by  definition,  a  new  idea. 
But  new  ideas  are  rooted  in  the  knowledge  that  makes  the  new 
idea  conceivable  and  practical.  Part  of  Better  Buying  Power  3.0 
involves  increased  support  for  education  in  STEM  (science, 
technology,  engineering,  and  mathematics).  Our  educational 
system  provides  the  foundation  of  our  knowledge,  but  that  is 
just  the  beginning.  Experience,  exposure  to  a  wide  and  diverse 
range  of  technical  fields,  and  continuing  in-depth  study  are  all 
important.  For  the  more  exciting  areas  of  technical  innova¬ 
tion  today,  this  knowledge  is  increasingly  highly  specialized 
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and  deep.  I  recently  visited  the  Massachusetts  Institute  of 
Technology  and  spoke  to  researchers  in  the  fields  of  biologi¬ 
cal  process-based  materials  production,  novel  computational 
architectures,  and  autonomy.  These  are  areas  in  which  it  is  not 
possible  to  enable  innovation  unless  one  has  a  deep  knowledge 
of  the  science  and  associated  technology.  I  believe  that  we 
are  in  the  early  stages  of  some  explosive  growth  in  the  prod¬ 
ucts  that  these  and  other  technologies  will  make  possible,  but 
some  very  specialized  advanced  technology  work  will  have  to 
be  accomplished  to  achieve  that  potential.  Once  that  occurs, 
innovative  applications  of  these  technologies  will  be  created 
at  an  exponential  rate.  In  many  cases  today,  the  DoD  is  not 
the  primary  financial  supporter  of  the  relevant  work.  Never¬ 
theless,  the  DoD's  basic  research  program  still  represents  an 
important  contributor,  and  it  provides  a  basis  by  which  the 
DoD  can  shape  and  capitalize  on  new  technical  knowledge  as 
it  is  created.  By  reaching  out  to  nontraditional  sources,  such  as 
through  the  DIU-X,  the  DoD  intends  to  increase  its  knowledge 
of  the  possibilities  that  commercial  cutting  edge  technology 
can  offer  to  DoD. 

My  second  ingredient  is  freedom.  By  this,  I  mean  the  free¬ 
dom  to  have  a  new  idea  and  to  take  action  in  pursuit  of  that 
idea.  I  mean  the  freedom  to  fail  and  start  again.  I  also  mean 
freedom  from  bureaucratic  constraints.  Our  free  enterprise 
system  provides  this  ingredient  on  a  national  scale,  and  it  is 
the  most  powerful  economic  engine  ever  created.  The  United 
States  stands  out  as  a  place  where  it  is  amazingly  easy  to  start 
a  new  business.  I've  done  it  a  couple  of  times. 

Within  the  DoD,  one  of  our  most  effective  and  successful  in¬ 
stitutions— the  Defense  Advanced  Research  Projects  Agency 
(DARPA)— is  a  living  testament  to  the  value  of  freedom.  I  zeal¬ 
ously  guard  DARPA's  freedom  from  the  many  parts  of  the  DoD 
that  see  DARPA's  budget  as  an  opportunity  to  fund  something 
they  need.  The  whole  concept  of  DARPA  is  that  the  organiza¬ 
tion  has  the  freedom  to  choose  its  own  high-risk  but  high- 
payoff  investments. 

In  DoD  more  broadly,  we  set  strategic  goals  for  technology  in¬ 
vestment,  require  a  certain  fraction  of  the  Services  Science  and 
Technology  work  to  be  in  these  areas  and  leave  those  organi¬ 
zations  the  freedom  to  choose  their  own  priorities  for  the  bal¬ 
ance  of  their  work.  Within  DoD,  we  also  allow  our  contractors 
to  pursue  Independent  Research  and  Development  (IR&D)  as 
an  allowable  overhead  cost  with  very  little  constraint. 

I  made  industry  a  little  nervous  recently  by  proposing  in  Bet¬ 
ter  Buying  Power  3.0  to  increase  the  DoD's  oversight  of  this 
work.  The  fundamental  concern  of  industry  partners  has  been 
the  possible  loss  of  freedom  to  make  their  own  IR&D  invest¬ 
ment  decisions.  That  was  never  my  intent.  I  once  ran  a  major 


defense  contractor's  IR&D  program,  and  I  appreciate  indus¬ 
try's  perspective.  I  appreciate  the  value,  to  industry  and  the 
DoD,  of  allowing  industry  to  place  its  own  bets  on  technology 
that  might  increase  a  firm's  competitiveness. 

After  carefully  considering  several  alternatives,  the  policy  I 
propose  would  merely  require  industry  to  brief  an  appropriate 
DoD  officer  or  official  prior  to  and  after  concluding  an  IR&D 
project,  and  to  document  that  the  meeting  occurred  as  part  of 
the  accounting  for  the  project.  This  policy  would  not  require 
sponsorship  or  approval  of  an  IR&D  project  by  a  DoD  official, 
but  it  would  require  industry  to  communicate  directly  with 
appropriate  DoD  personnel  and  to  obtain  feedback  on  the  pro¬ 
posed  work  and  to  communicate  the  results  when  the  work  is 
complete.  This  should  not  constrain  industry's  freedom  in  any 
way  that  current  regulations  and  statutes  don't  already  require, 
and  it  will  provide  the  benefit  of  ensuring  more  frequent  and 
effective  communication  between  industry  and  government. 

Human  Intangibles 

My  next  two  ingredients  enter  the  area  of  what  I  will  call  subjec¬ 
tive  human  intangibles.  These  intangibles  also  are  manifested 
in  what  we  call  organizational  cultures.  One  could  generate  a 
pretty  long  list  of  the  human  qualities  needed  for  successful 
innovation.  The  list  might  include  innate  intelligence,  creativity 
or  the  ability  to  think  "out  of  the  box"  and  curiosity,  to  name 
just  a  few  such  qualities.  These  address  the  capacity  to  have 
a  new  idea.  A  great  deal  of  work  has  gone  into  structuring  or¬ 
ganizational  environments  to  encourage  and  foster  creativity. 
This  can  include  physical  arrangements,  workplace  layouts, 
and  a  range  of  approaches  intended  to  foster  cultural  norms 
that  support  creativity. 

Some  companies  use  problem-solving  tests  to  identify  candi¬ 
dates  with  high  creativity.  I  believe  all  this  work  has  merit,  but 
I  also  think  its  goal  is  to  select  creative  people  and  to  draw  out 
the  inherent  creativity  that  people  either  do  or  do  not  possess. 
I'm  only  goingto  mention  two  human  qualities  that  I  think  have 
great  importance,  and  that  DoD  managers  at  all  levels  should 
be  especially  conscious  of:  risk  tolerance  and  persistence. 

Accepting  Risk 

I  was  asked  by  a  reporter  during  an  interview  2  or  3  years 
ago  if  the  DoD  was  taking  too  much  risk  in  its  programs. 
My  response  was  that  we  are  not  taking  enough  risks.  With 
respect  to  our  major  programs,  I  find  myself  pushed  in  two 
directions  simultaneously  by  the  political  winds  in  Wash¬ 
ington.  At  the  same  time  that  I  am  told  the  expectation  for 
all  our  programs  is  to  have  no  schedule  slips  or  cost  over¬ 
runs,  I  also  am  told  that  we  should  go  much  faster  in  our 
programs  and  not  have  so  much  oversight.  I'm  sorry,  but 
you  can't  have  it  both  ways. 
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My  observation  is  that  the 
politicization  of  these  decisions 
does  not  generally  lead  to  better 
results.  We  also  have  frequent 
leadership  changes— which 
makes  persistence  in  the  face  of 
difficulties  more  problematic. 


To  me,  both  perspectives  miss  the  point.  Development  of  new 
products,  particularly  a  new  generation  of  cutting-edge  and 
militarily  dominant  systems,  cannot  be  made  risk  free.  If  we 
want  risk-free  defense  acquisition,  we  should  just  buy  fully 
developed  products  from  other  countries.  If,  on  the  other  hand, 
we  want  the  best  military  in  the  world,  and  one  in  which  our 
warfighters  always  have  innovative  and  dominant  equipment, 
then  we  are  going  to  have  risk  in  our  programs. 

One  of  our  program  managers'  most  important  responsibilities 
is  to  understand  and  proactively  manage  the  risk  inherent  in 
any  development  program.  (I  wrote  about  that  responsibility 
in  an  article  in  the  July-August  2015  issue  of  Defense  AT&L 
magazine.)  To  borrow  a  line  from  the  movies,  the  secret  of 
life  is  balance.  We  have  to  balance  risk  against  urgency  and 
resource  constraints.  If  we  are  too  cautious,  our  programs 
will  take  forever  and  be  too  modest  in  their  ambitions.  If  we 
gamble  wildly,  we  will  waste  precious  resources  and  not  meet 
our  objectives. 

At  the  enterprise  level  in  DoD  today,  there  is  strong  support  for 
acceptingthe  risk  of  embarking  on  a  number  of  what  I  will  call 
advanced  technology  demonstration  programs.  The  recently 
completed  Long  Range  Research  and  Development  Planning 
Program  has  recommended  several  advanced  technology 
demonstration  programs  for  consideration  in  the  Fiscal  Year 
(FY)  2017  budget.  Similarly,  the  Strategic  Capabilities  Office 
is  proposing  demonstration  programs  based  on  novel  appli¬ 
cations  of  currently  fielded  systems  or  those  in  development. 
In  the  FY  2016  budget,  I  was  able  to  secure  funding  for  the 
Aerospace  Innovation  Initiative  that  will  culminate  in  X-plane- 
type  and  propulsion  technology  demonstrators  that  will  create 
options  for  the  systems  subsequent  to  our  current  Joint  Strike 
Fighter  program.  This  fall,  all  of  these  demonstration  propos¬ 
als  will  collide  with  budget  reality  at  the  President's  Budget 
request  level.  Needless  to  say,  if  sequestration  occurs,  that 
collision  will  be  even  more  violent.  In  some  cases,  we  could 
reasonably  accept  more  risk  and  move  directly  into  Engineer¬ 


ing  and  Manufacturing  Development  (EMD)  programs  instead 
of  pursuing  concept  demonstration  programs,  but  we  simply 
don't  have  the  resources  to  conduct  those  EMD  programs. 

Persistence 

The  other  intangible  characteristic  successful  innovators  dem¬ 
onstrate  is  persistence.  When  innovators  encounter  obstacles, 
they  find  ways  through  or  around  them.  Two  obvious  historical 
examples  are  Thomas  Edison  and  his  quest  for  a  practical  light 
bulb,  and  the  Wright  brothers  and  their  pursuit  of  controlled, 
powered  flight.  (David  McCullough  has  written  a  new  book 
chronicling  the  Wright  brothers'  tenacious  pursuit  of  powered 
and  controlled  flight.) 

The  DoD  has  sometimes  been  criticized  for  sticking  with  pro¬ 
grams  that  encounter  problems.  The  F-35  fighter  is  a  current 
example.  Earlier  ones  in  my  experience  include  the  C-17,  the 
Advanced  Medium-Range  Air-to-Air  Missile,  and  the  F-18E/F 
fighter.  In  all  those  cases,  we  persevered  and  achieved  good  re¬ 
sults.  In  other  cases,  we  have  stopped  programs  that,  in  retro¬ 
spect,  we  probably  should  have  continued.  In  still  other  cases, 
we  kept  going  for  far  too  long  on  programs  that  should  have 
been  canceled  earlier.  In  general,  my  sense  is  that,  for  most 
programs,  we  can  get  to  a  product  that  meets  our  require¬ 
ments  if  we  have  the  patience  and  persistence  to  continue. 
There  are  exceptions,  however. 

There  is  an  important  difference  between  the  persistence  ap¬ 
plied  to  commercial  innovation  and  that  applied  to  innovative 
products  in  DoD.  For  commercial  products,  both  in  start-ups 
and  large  corporations,  the  decision  to  continue  product  de¬ 
velopment  when  problems  are  encountered  is  driven  by  the 
judgment  of  the  management  (influenced  by  persistence  and 
risk  tolerance)  and  by  the  resources  available  to  the  firm.  In 
DoD's  case,  these  decisions  have  a  high  political  content— both 
internally  and  externally.  My  observation  is  that  the  politiciza¬ 
tion  of  these  decisions  does  not  generally  lead  to  better  results. 
We  also  have  frequent  leadership  changes— which  makes  per¬ 
sistence  in  the  face  of  difficulties  more  problematic.  I  have  no 
solution  to  offer  for  all  this  other  than  to  continue  the  work  of 
the  last  several  years  to  ensure  we  don't  start  unaffordable 
programs,  and  to  manage  risk  professionally  and  proactively 
in  our  development  programs.  The  DoD  spends  taxpayer-pro¬ 
vided  money;  we  will  always  be  under  close  public  scrutiny, 
and  we  will  always  have  internal  competition  for  resources. 

Collaboration 

Innovation,  in  the  commercial  and  the  DoD  context,  tends 
to  be  based  on  collaboration.  Multiple  technical  disci¬ 
plines  often  have  to  come  together,  and  the  synergy  be¬ 
tween  multiple  disciplines  may  be  the  central  feature  of  the 
innovative  idea.  In  the  DoD,  technical  ideas  only  reach  the 
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market  when  the  using  military  Service  decides  to  embrace 
the  new  concept  or  new  product.  This  is  not  quite  the  same 
as  the  commercial  market  where  "early  adopters"  from  a  large 
customer  base  may  help  a  technology  establish  a  foothold 
and  gain  credence.  Commercial  entrepreneurs  build  the  better 
mouse  trap  first  and  expect  customers  to  come.  In  DoD  the 
customers,  the  military  Departments,  ask  for  fairly  specific 
products  and  then  budget  the  resources  to  pay  for  the  devel¬ 
opment  of  those  products. 

The  DoD  also  uses  a  formalized  requirements  process  that  is 
based  on  the  perception  of  "gaps"  in  capability.  Requirements 
are  generated  to  fill  these  perceived  gaps.  This  approach  tends 
to  be  self-limiting  and  to  discourage  new  concepts  and  innova¬ 
tive  approaches  that  deviate  from  existing  paradigms.  Henry 
Ford's  famous  quip  that  if  he  had  asked  his  customers  what 
they  wanted  it  would  have  been  a  better  horse  has  some  rel¬ 
evance  here.  The  fact  is,  however,  that  despite  our  formal  pro¬ 
cess,  requirements  are  often  based  on  the  priorities  of  senior 
Service  leadership.  For  this  reason,  I  welcome  the  initiative 
from  the  U.S.  Senate  to  increase  Service  leadership  involve¬ 
ment  in  acquisition. 

A  strong  collaboration  between  Service  leadership  and  the 
technical  acquisition  community,  starting  as  early  in  the  prod¬ 
uct  life  cycle  as  possible,  is  essential  to  effective  innovation 
in  the  DoD,  and  it  is  a  component  of  Better  Buying  Power.  I 
would  also  add  that  close  collaboration  with  the  intelligence 
community  is  critical  as  well:  Potential  adversaries  are  moving 
very  quickly  to  develop  products  clearly  designed  to  defeat 
U.S.  capabilities.  The  DoD  must  be  both  innovative  and  quick 
to  market  in  responding  to  these  emerging  threats.  Achieving 
these  objectives  requires  strong  and  continuous  collabora¬ 
tion  between  operators,  the  intelligence  community  and  the 
technical  acquisition  community. 

Funding  Is  Fundamental 

There  is  one  more  necessary  ingredient  that  I  have  not  dis¬ 
cussed  yet.  That  ingredient  is  capital.  Small  start-ups  and  large 
businesses  alike  depend  on  capital  to  survive  and  to  bring  new 
products  to  market.  So  it  is  for  the  DoD,  and  this  is  my  greatest 
concern  today.  Our  capital  comes  from  the  budgets  we  receive 
from  Congress.  As  long  as  we  remain  trapped  in  the  grip  of 
sequestration  and  as  long  we  continue  to  prepare  budgets  that 
are  far  out  of  alignment  with  the  funds  we  may  receive,  we  will 
not  be  able  to  innovate  effectively. 

Innovation  isn't  just  about  thinking  outside  the  box,  or  about 
demonstrating  new  technologies  and  operational  concepts.  It 
is  about  developing,  producing,  fielding  and  training  with  those 
new  capabilities.  Today  I  believe  our  pipeline  of  new  products 
in  development  is  inadequate  to  deal  with  emerging  threats. 


We  are  facing  a  major  recapitalization  bill  for  the  strategic 
deterrent  that  is  about  to  come  due.  There  is  nothing  that  I  or 
the  DoD  can  do  to  improve  our  productivity  and  efficiency  that 
will  fully  compensate  for  inadequate  capital.  All  the  efficiencies 
I  can  even  imagine  will  not  make  up  this  shortfall.  By  conduct¬ 
ing  well-chosen  demonstrations,  we  can  reduce  the  lead  time 
to  acquiring  real  operational  capability,  we  can  keep  an  es¬ 
sential  fraction  of  our  industrial  base  gainfully  employed,  and 
we  can  position  ourselves  for  changes  in  threat  perceptions 
and  the  availability  of  additional  funds.  But,  without  relief  from 
the  specter  of  sequestration,  we  cannot  increase  the  relative 
combat  power  of  the  United  States  against  our  most  capable 
potential  adversaries. 

I  can  point  to  numerous  places  in  DoD  where  we  are  taking 
steps  to  improve  our  access  to  and  use  of  each  of  these  in¬ 
gredients:  knowledge,  freedom,  risk  tolerance,  persistence, 
collaboration  and  capital.  For  the  last  few  years,  we  have 
worked  hard  to  emphasize  and  increase  the  professionalism 
of  the  government  acquisition  workforce.  Secretary  Carter's 
"Force  of  the  Future"  initiative  is  specifically  intended  to  bring 
high  knowledge  people  into  our  workforce.  With  help  from 
the  Congress  through  the  Defense  Acquisition  Workforce 
Development  Fund  and  a  number  of  internal  actions,  we  have 
continued  to  build  on  our  strong  foundation  in  this  area  despite 
budget  constraints. 

We  are  protecting  and  emphasizing  the  freedom  of  our  man¬ 
agers  to  find  creative  solutions  to  technical  and  managerial 
problems.  Last  year,  I  tasked  each  of  our  program  managers 
to  communicate  directly  with  me  about  problems,  issues  and 
recommended  solutions.  The  result  was  a  huge  testament  to 
the  creativity,  dedication  and  professionalism  of  our  workforce. 

The  demonstrations  that  I  mentioned,  if  they  can  be  funded, 
show  our  willingness  to  take  risk  on  new  and  nontraditional 
approaches  to  operational  problems.  Deputy  Secretary  Work's 
"Third  Offset"  strategy,  by  its  very  nature,  will  require  the  DoD 
to  accept  the  risk  associated  with  new  operational  concepts 
and  the  technologies  that  enable  them.  Our  ability  to  persist 
in  bringing  all  of  these  initiatives  to  fruition  remains  to  be  seen, 
but  the  closely  aligned  leadership  in  the  DoD— including  the 
Secretary  and  Deputy  Secretary  of  Defense,  myself,  and  the 
new  Joint  and  Service  uniformed  chiefs— makes  me  optimistic 
that  we  can  collaborate  to  do  so. 

From  their  inception,  the  Better  Buying  Power  initiatives,  in 
every  edition,  have  been  about  getting  the  most  value  pos¬ 
sible  from  our  available  capital.  With  that  possible  exception— 
which  is  in  the  hands  of  the  Congress— we  possess  or  can 
obtain  all  the  ingredients  we  need  to  bring  innovative  solutions 
to  our  warfighters.  & 
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More  than  2  years  ago,  I  left  a  job 
I  loved— teaching  at  the  Defense 
Acquisition  University— for  an  op¬ 
portunity  to  work  in  the  Office  of 
Performance  Assessments  and 
Root  Cause  Analyses  (PARCA).  My  primary  re¬ 
sponsibility  is  to  conduct  root  cause  analyses  of 
troubled  DoD  acquisition  programs— those  that 
have  undergone  a  critical  Nunn-McCurdy  breach,  or 
others  as  assigned  by  the  Under  Secretary  of  Defense 
for  Acquisition,  Technology,  and  Logistics  (USD[AT&L]>  or  by 
the  Secretaries  of  the  Military  Departments. 

Many  colleagues  asked  why  I  would  want  to  work  in  the  Pentagon,  and  there 
was  one  main  reason:  Odd  as  it  may  sound,  I  am  fascinated  by  acquisition 
cost  growth.  Believe  it  or  not,  cost  analysts  actually  get  together  and  debate 
these  kind  of  things— very  passionately! 


Husband  is  the  Senior  Advisor  for  Root  Cause  Analyses  in  the  Office  of  Performance  Assess¬ 
ments  and  Root  Cause  Analyses  (PARC AX  part  of  the  Office  of  the  Undersecretary  of  Defense 
for  Acquisition,  Technology,  and  Logistics.  He  is  a  retired  Air  Force  officer  with  a  doctorate  in 
chemical  engineering  from  Germany's  Karlsuhe  Institute  of  Technology. 
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There 
are  three 
relatively  new 
forces  in  play  that  have  had 
profoundly  positive 
influences  on 
acquisition  program 
results:  one  statutory,  one 
regulatory,  and  one  force 
related  to  DoD  culture. 


have  strong  opinions  about  key  factors  that  may  be  responsible 
for  recent  improvements. 

In  my  view,  there  are  three  relatively  new  forces  in  play  that 
have  had  profoundly  positive  influences  on  acquisition  pro¬ 
gram  results:  one  statutory,  one  regulatory,  and  one  force 
related  to  DoD  culture.  The  Weapon  Systems  Acquisition 
Reform  Act  (WSARA)  of  2009  is  a  statutory  change  that  has 
been  lauded  by  most  defense  experts  as  extremely  positive. 
WSARA  made  sweeping  changes  that  have  strengthened  ac¬ 
countability  of  acquisition  execution  and  oversight  officials  and 
ensured  that  programs  are  started  with  more  realistic  cost  and 
schedule  baselines,  performance  expectations,  and  mature 
technologies.  In  particular,  several  WSARA-related  changes 
have  led  to  increased  focus  on  assessing  a  program's  readiness 
for  initiation  prior  to  Milestone  (MS)  B,  including  requirements 
for  an  Independent  Cost  Estimate  for  major  programs  at  MS 
A,  a  Preliminary  Design  Review  prior  to  MS  B,  and  measures 
to  ensure  adequate  competition,  including  competitive  proto¬ 
typing,  dual  sourcing,  and  modular  open  architectures,  among 
others.  WSARA  also  strengthened  test  and  evaluation  and 
systems  engineering  functions  that  are  critical  to  Engineering 
and  Manufacturing  Development  by  establishing  statutory  di¬ 
rectors  of  those  offices  appointed  by  the  Secretary  of  Defense. 


It  is  impolitic  to  say  so  publicly,  but  I  looked  forward  to  the 
opportunity  to  examine  Nunn-McCurdy  programs  and  de¬ 
termine  what  caused  their  cost,  schedule  and  performance 
shortfalls.  Conducting  such  analysis  is  challenging  and  intel¬ 
lectually  stimulating,  and  for  a  defense  analyst  it  is  a  treasured 
opportunity  to  use  analytical  skills  and  expert  judgment  to 
create  a  product  that  will  be  scrutinized  intensely  (particularly 
by  those  responsible  for  program  execution,  who  may  take 
issue  with  the  conclusions).  Products  of  these  reviews  also 
can  have  a  positive  impact,  through  the  dissemination  of  les¬ 
sons  learned.  In  2013,  a  commonly  expressed  fear  "inside  the 
building"  was  that  sequestration  would  result  in  a  plethora  of 
Nunn-McCurdy  breaches  due  to  the  impending  reduction  of 
production  quantities  and  stretched-out  schedules. 


WSARA  improved  the  DoD's  assessment  of  troubled  acquisi¬ 
tion  programs  by  establishing  the  PARCA  office,  which  con¬ 
ducts  root  cause  analyses  of  critical  Nunn-McCurdy  breach 
programs  and  follow-on  performance  assessments  of  those 
programs  in  an  effort  to  prevent  future  cost  and  schedule 
growth.  Finally,  WSARA  increased  the  penalty  on  critical 
Nunn-McCurdy  programs  by  rescinding  those  programs'  most 
recent  milestone  approval  and  adding  a  "presumption  of  ter¬ 
mination"  unless  the  more  stringent  certification  criteria  are 
met.  Changes  instantiated  by  WSARA  could  not  be  expected 
to  have  immediate  impacts  (because  of  the  number  of  pre- 
WSARA  programs  in  the  pipeline),  but  I  believe  the  positive 
impacts  of  WSARA  now  are  clearly  evident,  particularly  for 
programs  initiated  since  2010. 


So  what  has  happened  in  the  last  2  years?  The  Department  of 
Defense  (DoD)  has  had  a  total  of  two  critical  Nunn-McCurdy 
programs  for  which  a  root  cause  analysis  is  required  (and  one 
other  that  resulted  from  program  termination).  This  compares 
to  an  average  of  4.5  per  year  from  2006  through  2011.  Counting 
all  Nunn-McCurdys  (both  critical  and  significant  breaches), 
the  DoD  had  an  average  of  2.3  per  year  from  2012  through 
2014,  compared  to  an  average  of  6.3  per  year  from  2006 
through  2011. 

Coming  up  with  explanations  for  causes  of  improvement  is 
even  trickier  than  coming  up  with  root  causes  of  problems. 
While  opinions  abound,  it  is  difficult  to  ascertain  that  a  given 
action  or  set  of  actions  is  responsible  for  an  observed  result, 
particularly  for  something  as  complex  as  DoD  acquisition  pro¬ 
grams,  which  have  so  many  internal  and  external  influences. 
Analytical  rigor  notwithstanding,  I,  like  many  defense  analysts, 


Second,  regulatory  changes  that  I  believe  have  had  demonstra¬ 
bly  positive  impacts  on  program  results  are  the  Better  Buying 
Power  (BBP)  initiatives  begun  by  Secretary  of  Defense  Ash¬ 
ton  Carter  in  2010  when  he  was  the  USD(AT&L).  Acquisition 
insiders  are  aware  of  the  broad  scope  of  the  BBP  initiatives, 
which  have  the  overarching  goal  of  strengthening  the  DoD's 
buying  power,  improving  productivity,  and  providing  more  af¬ 
fordable  products  to  the  warfighters.  Carter  and  his  successor 
as  USD(AT&L),  Frank  Kendall,  have  emphasized  that  BBP  is 
not  acquisition  reform  but  instead  a  continuous  process  of 
improving  how  we  acquire  goods  and  services  based  on  proven 
methods  and  analysis  of  data. 

Because  two  successive  Defense  Acquisition  Executives  have 
demonstrated  such  a  clear  and  lasting  commitment  to  im¬ 
proving  the  acquisition  system  through  BBP,  I  think  that  BBP 
concepts  and  principles  have  been  successfully  infused  into 
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the  acquisition  workforce.  In  my  view,  while  the  individual  ini¬ 
tiatives  provide  important  guidance  about  key  principles,  an 
equally  important  contribution  of  BBP  has  been  the  dialogue  it 
has  fostered  within  the  workforce,  engaging  the  entire  defense 
acquisition  community  to  think  about  what  works  and  how  to 
do  things  better. 

Finally,  while  those  outside  the  DoD  may  not  have  seen  it,  I 
believe  a  third  key  factor  responsible  for  improved  acquisi¬ 
tion  performance  is  that  DoD  culture  has  changed  (or  at  least 
shifted).  More  than  at  any  time  in  my  career,  cost  and  afford¬ 
ability  are  emphasized  by  leadership  as  key  considerations  for 
the  goods  and  services  that  the  DoD  procures. 

All  acquisition  professionals  are  aware  that  three  things  are 
supposed  to  be  balanced  (or  traded)  in  an  acquisition  pro¬ 
gram-cost,  schedule  and  performance.  For  many  decades, 
performance  was  king;  system  requirements  and  designs 
pushed  the  performance  envelope  toward  development  of 
what  former  Secretary  of  Defense  Robert  Gates  called  "ex¬ 
quisite  systems,"  at  the  expense  of  cost  and  schedule.  Begin¬ 
ning  in  the  late  2000s,  that  mentality  began  shifting,  and  cost 
considerations  entered  into  discussions  more  than  before.  I 


think  congressional  changes  in  2006  to  the  Nunn-McCurdy 
law  were  an  early  stimulator  of  this  change:  No  longer  was  it 
possible  to  simply  rebaseline  troubled  programs  without  pen¬ 
alty.  Further  changes  to  the  law  that  increased  the  stringency 
of  the  Nunn-McCurdy  process  made  congressional  intent 
crystal  clear— lawmakers  have  lost  patience  with  programs 
that  don't  keep  their  cost  and  schedule  promises.  I  believe 
the  DoD's  shift  to  a  more  "cost-conscious"  culture  also  is 
a  byproduct  of  the  WSARA  reforms  and  BBP.  Not  only  is  it 
a  specific  BBP  initiative,  it  also  is  woven  throughout  BBP's 
overarching  goals  of  providing  more  affordable  products  and 
improving  productivity. 

Obviously,  looking  at  Nunn-McCurdy  programs  through  the 
prism  of  the  last  2  years  is  not  an  analytically  rigorous  basis 
for  drawing  hard  and  fast  conclusions.  And  it  could  well  be 
that  continuing  budget  gridlock  ultimately  will  cause  more  pro¬ 
grams  to  scale  back,  retrench  and  stretch  out.  As  the  DoD's 
"Maytag  repairman"  for  root  cause  analyses  looking  for  work,  I 
can  only  hope.  But  for  the  American  taxpayer's  sake,  we  must 
instead  wish  that  the  DoD's  trend  of  decreasing  numbers  of 
Nunn-McCurdys  continues. 

The  author  can  be  reached  at  david.m.husband.civ@mail.mil. 
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The  Defense  Acquisition  University  (DAU)  has  been  training  Department  of  De¬ 
fense  (DoD)  program  managers  (PMs)  for  more  than  40  years.  During  that  time, 
the  training  requirements  have  changed  considerably.  Most  basic  courses  are 
now  done  online,  while  intermediate  courses  feature  a  combination  of  online 
prerequisites  followed  by  classroom  training.  Advanced  PM  courses  use  teams 
of  students  to  discuss  current  acquisition  issues,  analyze  and  prepare  case  studies,  share 
lessons  from  their  experience  and  provide  feedback  to  their  colleagues. 

Over  the  years,  the  duration  and  content  of  DoD's  PM  training  has  evolved  from  the  initial  5-month  program 
management  course  to  a  series  of  shorter  courses  taken  along  a  PM's  career  path.  The  current  PM  training 
framework  is  shown  in  Figure  1.  To  reach  Level  III  certification  requires  346  hours  of  online  instruction  and  27.5 
classroom  days.  For  major  acquisition  PMs,  70  additional  classroom  days  of  specialized  training  (PMT  401  and 
PMT  402)  are  required.  The  higher  levels  of  certification  require  significant  training  along  with  relevant  educa¬ 
tion  and  work  experience. 


Gadeken  is  a  Professor  at  the  Defense  Acquisition  University  at  Fort  Belvoir,  Virginia.  His  current  interest  is  helping  program  managers  become 
effective  leaders.  Gadeken  received  his  doctorate  in  engineering  management  from  the  George  Washington  University. 


Defense  AT&L:  November- December  2015 


10 


This  article  concentrates  on  the  executive-level  courses 
(PMT  401  and  402)  used  to  train  PMs  who  will  lead  the 
most  expensive,  critical  and  highly  visible  programs  in 
the  DoD.  These  training  programs  are  conceived,  devel¬ 
oped  and  taught  by  DAU  faculty  with  only  infrequent  use 
of  vendors  or  outside  resources. 

Designed  to  improve  DoD  acquisition  outcomes,  the  10- 
week  Program  Manager's  Course  (PMT  401)  seeks  to: 

■  Enhance  critical  thinking  and  decision-making  skills. 

■  Develop  the  capability  to  lead  cross-functional  inte¬ 
grated  product  teams  in  an  acquisition  environment. 

■  Embed  the  habits  of  reflection,  feedback  and  con¬ 
tinuous  learning. 

The  PMT  402  Executive  PM  Course  is  an  assignment- 
specific  course  for  students  being  assigned  to  lead 
major  acquisition  programs.  The  crux  of  this  course  is 
for  each  student  to  present  an  analysis  of  the  student's 
new  program  and  then  work  with  a  team  of  students 
and  faculty  to  develop  an  action  plan  once  they  are 


assigned.  With  this  in  mind,  the  remainder  of  this  article 
concentrates  on  how  DAU  implements  the  three  PMT 
401  core  themes  of  critical  thinking  and  decision  making, 
leadership  skills,  and  habits  of  learning. 

Critical  Thinking  Skills 

Managing  complex  defense  programs  requires  both 
subject-matter  expertise  and  disciplined  thinking.  As 
stated  by  our  current  Defense  Acquisition  Executive, 
Frank  Kendall: 

The  first  responsibility  of  key  leaders  in  the  acquisition 
workforce  is  to  think.  One  of  the  many  reasons  that  our 
key  leaders  have  to  be  true  professionals  who  are  fully 
prepared  to  do  their  jobs  by  virtue  of  their  education, 
training  and  experience  is  that  creative,  informed  thought 
is  necessary  to  optimize  the  structure  of  a  program. 

DAU  embeds  principles  of  critical  thinking  in  each  of 
our  PM  courses,  and  the  executive  courses  use  criti¬ 
cal  thinking  elements,  standards  and  intellectual  traits 
from  Richard  Paul  and  Linda  Elder's  "Aspiring  Thinker's 
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Figure  1.  DoD  Program  Management  Certification  Requirements 


Level  I  Certification  Level  II  Certification 


Level  III  Certification 


ACQ 101 

Fundamentals  of  Systems 
Acquisition  Management 

25  hrs.,  online 

ENG  101 

Fundamentals  of 
Systems  Engineering 

35  hrs.,  online 


CLB007 

Cost  Analysis 

CLV  016 

Introduction  to  Earned 
Value  Management 

1  Year  of  Experience 


ACQ  201A 

Intermediate  Systems 
Acquisition,  Part  A 

ACQ  201 B 

Intermediate  Systems 
Acquisition,  Part  B 

25  hrs.,  online 

4.5  days  classroom 

PMT  251 

Program  Management 
Tools  Course,  Part  1 

PMT  257 

Program  Management 
Tools  Course,  Part  II 

20  hrs.,  online 

4.5  days  classroom 

CON  121 

Contract  Planning 

CON  124 

Contract  Execution 

12  hrs.,  online 

12  hrs.,  online 

CON  127 

Contract  Management 

10  hrs.,  online 

IRM  101 

Basic  Information 
Systems  Management 

EVM  101 

Fundamentals  of  Earned 
Value  Management 

34  hrs.,  online 

34  hrs.,  online 

2  Years  of  Experience 


PMT352A  PMT352B 

Program  Management  Program  Management 

Office  Course,  Part  A  Office  Course  Part  B 

22  hrs.,  online  18.5  days  classroom 


LOG  103 

Reliability,  Availability 
and  Maintainability 

32  hrs.,  online 


PMT-401  and  PMT-402  meet  statutory  requirements  for 
PEO/ACAT  I/ll  PM  &  Deputy  PM  (Title  10  U.S.C.  1735) 

PMT  401  PMT  402 

Program  Manager's  Executive 

Course  PM  Course 

10  weeks  classroom  20  days  classroom 

+  online  modules 

*  Potential  ACAT  I,  IA,  II  and  III  PMs,  Dpty  PMs 

*  PEOs  &  ACAT  I,  KA,  II  PMs  &  Dpty  PMs 


SYS  202 

Intermediate  Systems 
Planning,  Research 
Development  and 
Engineering 

9  hrs.,  online 


BCF103 

Fundamentals  of  Business 
Financial  Management 

26  hrs.,  online 


Guide  to  Critical  Thinking.”  As  an  example,  standards  of  critical 
thinking  include  clarity,  accuracy,  significance,  completeness 
and  fairness. 

Applying  the  proven  doctrine  of  "train  as  you  fight,”  the  PMT 
401  curriculum  uses  program  management  case  studies  to 
develop  critical  thinking  and  decision-making  skills.  These  case 
studies  are  prepared  by  faculty  and  students  and  are  based  on 
real  defense  programs.  The  10-week  PMT  401  course  includes 
more  than  80  case  studies  that  cover  a  broad  spectrum  of  ac¬ 
quisition  life-cycle  development  and  sustainment  issues.  Each 
case  provides  the  background  on  a  program  and  then  presents 
the  students  with  a  dilemma  (or  problem)  that  the  PM  needs 
to  address.  Students  analyze  the  case  and  then  present  and 
debate  alternatives  for  how  the  PM  should  respond. 


4  Years  of  Experience 


faculty  sponsor  for  the  case,  who  often  is  the  case  author  as 
well.  After  class,  students  have  an  opportunity  to  reflect  on 
what  they  learned  and  are  given  a  learning  journal  in  which  to 
record  their  reflections.  Faculty  small  group  advisors  prompt 
students  to  share  their  reflections  as  the  course  proceeds. 

Faculty  members  are  encouraged  to  "not  teach”  but  to  facili¬ 
tate  their  cases.  Case  studies  normally  have  no  "right”  answer, 
so  the  emphasis  is  on  the  analysis  and  critical  thinking  skills 
demonstrated  by  the  students.  The  case  studies  also  are  used 
to  apply  a  variety  of  management  tools  and  frameworks  such 
as  stakeholder  analysis,  interest-based  negotiations,  polarity 
thinking,  appreciative  inquiry  and  action  learning. 


A  very  significant  part  of  the  case  learning  process  is  the 
four-stage  learning  model  shown  in  Figure  2.  Following  this 
sequential  process  is  quite  important  if  one  is  to  learn  as  much 
as  possible  from  each  case.  The  first  step  of  individual  prepara¬ 
tion  occurs  when  students  read  the  case— normally  during  the 
evening  prior  to  the  class.  Second,  small  groups  of  five  to  seven 
students  meet  in  the  morning  before  class  starts  and  briefly 
discuss  each  case  with  a  faculty  facilitator.  Third,  all  students 
convene  in  the  classroom  for  a  facilitated  discussion  led  by  the 


Near  the  end  of  the  course,  each  student  team  prepares  its 
own  case  study  based  on  one  of  the  team  member's  real-world 
dilemmas  and  then  facilitates  the  case  with  the  rest  of  the 
class.  This  is  not  only  a  very  popular  part  of  the  course  but 
has  become  our  leading  approach  for  new  case  development. 

Leadership  Skills 

If  case  studies  are  the  heart  of  the  Program  Manager's  Course, 
leadership  is  the  soul  of  the  course.  Students  arrive  in  class 
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Figure  2.  Case  Study  Four-Stage  Learning  Process 


Time 


with  a  portfolio  of  tech¬ 
nical  and  management 
skills,  so  it  is  often  their 
leadership  skills  that  most 
need  development.  This 
is  illustrated  in  the  career 
development  timeline  in 
Figure  3.  Technical  and 
management  skills  are 
still  required  for  these 
PMs,  but  it  is  the  leader¬ 
ship  skills  that  will  most 
influence  their  ultimate 
success. 

In  "Marching  an  Army 
Acquisition  Program 
Toward  Success"  ( De¬ 
fense  AT&L,  November- 
December  2012),  the 
authors  found  that  "the 
first  characteristic  that 
separates  the  really  suc¬ 
cessful  PMs  is  their  leadership.  They  set  the  tone,  they  should 
be  decisive,  and  have  a  vision." 

The  leadership  theme  also  was  stressed  by  Kendall  in  the  May- 
June  2013  issue  of  Defense  AT&L: 

Having  seen  more  than  4  decades  of  defense  acquisition  policy 
changes,  I  am  absolutely  convinced  that  nothing  matters  as 
much  as  professional  leadership.  Once  you  have  that,  the  rest 
is  details. 

At  DAU,  an  assessment-based  approach  fosters  leadership 
development:  Instead  of  teaching  leadership  skills,  the  course 
focuses  on  assessing  these  skills  so  our  students  can  work  on 
their  own  development  while  they  are  with  us  and  after  they 
return  to  the  workplace.  Various  assessment  tools  and  team 
exercises  are  employed  to  highlight  students'  strengths  as  well 
as  their  leadership  development  needs. 


The  assessment  starts  before  students  arrive  by  having  them 
complete  online  personality  type,  emotional  intelligence,  and 
workplace  (360-degree)  assessments.  The  reports  then  are 
provided  to  students  early  in  the  course  along  with  the  op¬ 
portunity  to  share  and  discuss  them  with  both  peers  and  se¬ 
lected  faculty.  The  key  is  to  integrate  these  assessments  into 
a  framework  to  encourage  further  skill  development. 

The  assessment  continues  with  observation  of  students  in 
small-  and  large-group  case  discussions.  How  effectively 
do  students  communicate  their  thoughts,  build  on  others' 
comments,  and  help  the  group  reach  consensus  on  how  to 
address  each  case  dilemma?  In  addition  to  facilitating,  faculty 
members  must  observe  student  performance  and  provide 
feedback  to  students  both  individually  and  in  scheduled  small 
group  sessions. 

Team  exercises  and  simulations  are  used  to  place  students 
in  a  team  environment  where  they  get 
additional  feedback.  One  example  of 
this  is  "Looking  Glass,"  an  organizational 
simulation  that  DAU's  Defense  Systems 
Management  College  licenses  from  the 
Center  for  Creative  Leadership.  Here, 
class  members  take  the  key  leadership 
positions  in  a  commercial  glass  manufac¬ 
turing  company  and  are  challenged  to  run 
the  company  for  an  entire  day.  After  the 
simulation,  a  second  day  is  dedicated  to 
feedback  on  how  each  team  and  individ¬ 
ual  performed  in  the  exercise  and  how  the 
feedback  supports  their  personal  leader¬ 
ship  development  goals.  There  also  is  a 
media  workshop  in  which  each  student  is 


Figure  3.  Career  Development  for  Program  Managers 
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Figure  4.  DAU  Acquisition  Learning  Model 


Give  the  workforce  Help  the  workforce 


Exploit  high-impact 


long-term  knowledge 
and  habits  of  mind 


succeed  on  the  job 
every  day 


opportunities  for  teams 
and  the  overall  workforce 


given  a  PM  role  from  one  of  the  case  studies  and  required  to 
explain  and  defend  his  or  her  program  to  members  of  the  press 
(with  the  press  role  performed  by  faculty  members).  These 
media  interviews  are  videotaped  and  analyzed  for  learning 
points  by  groups  of  students. 

The  crux  of  the  leadership  development  approach  is  that,  be¬ 
fore  students  can  lead  a  program  or  team,  they  must  be  able 
to  both  understand  and  lead  themselves. 

Reflection,  Feedback 
and  Continuous  Learning 

Effective  PM  training  is  more  than  a  series  of  course  modules 
filled  with  content.  There  are  key  learning  processes  that  lit¬ 
erally  make  or  break  a  training  event  or  program.  The  first  of 
these  is  establishing  a  positive  learning  climate  both  inside  and 
outside  of  the  classroom.  Nothing  can  be  more  counterpro¬ 
ductive  to  learning  than  for  students  not  to  want  to  be  there 
or  to  be  distracted  constantly  by  outside  events.  At  DAU,  vari¬ 
ous  approaches  are  used  to  create  a  collegial  atmosphere  and 
keep  the  focus  on  learning.  These  include  icebreaker  exercises, 
class  lunches  and  socials,  and  even  cookouts  and  competitive 
games  in  the  evenings.  The  goal  is  to  have  students  enjoy  being 
together  and  learning  together  while  developing  personal  and 
professional  relationships  to  draw  on  during  the  remainder  of 
their  careers. 

The  second  key  learning  process  is  team  learning.  Defense 
programs  are  managed  with  cross-functional  teams;  there¬ 
fore,  the  learning  approach  also  features  cross-functional 
teams.  Students  work  all  of  their  case  studies  in  teams 


(called  small  groups),  and  these  teams  are  remixed  halfway 
through  the  course  so  students  can  work  with  new  peers 
and  faculty  members.  Experience  demonstrates  the  team¬ 
work  approach  to  be  very  beneficial  during  the  courses. 
Students  frequently  add  these  peers  to  their  professional 
network  and  often  consult  with  them  on  program  issues 
once  they  return  to  the  workplace. 

The  third  key  learning  process  is  peer  feedback.  In  DAU's  se¬ 
nior  PM  courses,  there  are  no  tests,  no  term  papers  and  no 
grades.  The  key  process  for  student  evaluation  is  feedback 
given  to  each  student  by  both  peers  and  faculty  using  a  sim¬ 
ple  process  developed  by  the  Center  for  Creative  Leadership 
called  Situation-Behavior-Impact  (SBI).  As  outlined  in  the  Cen¬ 
ter's  publication  Feedback  That  Works ,  the  SBI  process  seeks 
to  overcome  conversational  feedback,  which  often  is  vague  or 
judgmental.  Instead  of  feedback  like  "great  job  on  that  report," 
one  might  say,  "The  program  status  report  you  submitted  yes¬ 
terday  [Situation]  contained  all  the  requested  data  and  your 
analysis  was  so  thorough  [Behavior]  that  the  boss  has  cited  it 
as  an  example  for  others  to  follow  [Impact]." 

To  implement  the  feedback,  students  share  their  personal 
learning  goals  with  their  small  group  and  have  fellow  students 
hold  each  other  accoutable  for  working  on  their  goals.  After 
the  SBI  process  is  explained  early  in  the  course,  peer  feedback 
sessions  are  scheduled  every  few  weeks  to  provide  students 
feedback  on  their  development  goals  while  allowing  them  to 
practice  providing  feedback  to  others.  This  increases  the  like¬ 
lihood  they  will  use  the  SBI  feedback  process  with  their  real 
project  teams  when  they  return  to  work. 
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For  PM  training  to  have  the  greatest  impact,  its  benefits  must 
continue  after  students  return  to  the  workplace.  Over  the 
years,  DAU  has  expanded  its  focus  from  being  just  a  training 
provider  to  becoming  a  broad-based  performance  support 
organization.  This  is  reflected  in  DAU's  Acquisition  Learning 
Model  illustrated  in  Figure  4. 

Today,  attending  a  training  program  is  just  the  starting  point  in 
developing  an  ongoing  relationship  with  DAU.  After  returning 
to  the  workplace,  graduates  have  a  network  of  faculty  mem¬ 
bers  they  can  call  on  to  assist  them  with  emerging  program 
issues.  DAU  also  has  a  large  set  of  online  references,  tools  and 
learning  modules  that  are  available  to  its  graduates.  For  larger 
problems,  a  faculty  team  can  be  assembled  to  provide  onsite 
team  training  or  mission  assistance. 

Executive  coaching  also  is  in  DAU's  repertoire  of  support 
tools.  With  more  than  50  faculty  members  formally  trained 
as  executive  coaches,  DAU  recently  won  a  national  award 
for  the  outcomes  achieved  by  its  enterprise-wide  executive 
coaching  program. 

What  this  means  for  PMs  is  that,  once  they  complete  an 
executive  course,  they  can  request  an  executive  coach  to 
help  them  with  their  newly  assigned  program.  The  coach 
provides  the  ideal  "sounding  board"  to  help  the  new  PM 
think  through  a  variety  of  program  issues,  including  devel¬ 
oping  a  program  plan,  reorganizing  or  dealing  with  difficult 
program  stakeholders. 

Comments  from  PMs  who  have  used  DAU  executive  coaches 
include,  "A  catalyst  for  change.  My  coach  challenged  me  to 
think  creatively  and  systematically.  I  couldn't  lead  the  way  I 
was  used  to  leading.  My  coach  helped  my  me  adapt  and  try  on 
new  behaviors,"  and  "A  strategic  thinking  partner.  Helped  me 
go  to  the  strategic  perspective.  At  my  level,  I  really  appreciated 
having  someone  who  knew  the  ropes  and  path,  but  didn't  tell 
me  what  to  do."  Customer  demand  for  the  coaching  program 
continues  to  grow. 

Summary 

With  the  return  of  troops  from  overseas  deployments,  the 
defense  budget  is  shrinking  and  likely  will  be  lower  in  future 
years.  Nevertheless,  defense  programs  continue  to  demand 
highly  talented  PMs  who  can  deliver  needed  products  and 
services  to  our  military  customers.  The  DAU's  challenge  is 
to  make  its  training  and  program  support  tools  even  more 
effective  in  this  resource-constrained  environment.  Anchor 
points  for  the  training  approach  are  enhancing  the  PMs' 
critical  thinking  and  decision-making  skills,  developing  their 
portfolios  of  leadership  skills,  and  embedding  the  habits  of 
reflection,  feedback  and  continuous  learning.  In  evolving  to 
meet  the  demand  for  effective  training,  DAU  must  continue 
to  adapt  and  change  so  it  can  develop  top  performing  PMs 
who  will  meet  our  warfighters'  even  more  stingent  demands 
to  keep  our  nation  safe  and  secure.  & 

The  author  can  be  reached  at  owen.gadeken@dau.mil. 
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For  the  last  8  years,  I  have 
been  a  faculty  member 
in  the  Defense  Acquisi¬ 
tion  University's  Executive 
Leadership  and  Coaching 
programs,  and  I  thought  it  would 
be  worthwhile  to  share  our  stu¬ 
dents'  definition  of  the  leader's 
role  in  establishing  and  running 
extraordinary  teams. 

The  journey  to  gather  the  data  started  with  a  classroom  discussion.  I  had 
arranged  for  a  guest  speaker  who  had  led  several  successful  programs, 
and  during  my  introduction  I  started  to  tell  a  story  about  when  he  assumed 
command  of  a  failing  missile  program.  One  of  his  first  acts  was  placing 
PowerPoint  slides  in  every  cubicle  that  read,  "Be  on  contract  by  July."  He 
continued  to  emphasize  this  point  over  the  next  few  days,  individually  ask¬ 
ing  each  team  member,  "How  is  what  you  are  working  on  right  now  helping 
this  team  get  on  contract  by  July?"  At  this  point,  a  Marine  Corps  colonel 
stopped  me  and  said:  "Scott,  this  is  Leadership  101.  Please  don't  tell  me  that 
is  what  we  will  be  focusing  on  this  morning." 

This  colonel  was  a  decorated  war  hero,  and  in  only  one  week  had  earned  the 
respect  of  his  peers  and  the  faculty.  He  was  a  leader  beyond  leadership  101. 
Or  was  he?  I  then  asked  him  what  his  program's  purpose  was.  Somewhat 
flustered,  he  gave  an  answer.  Next  I  asked,  "Who  are  three  people  on  your 
program  who  report  directly  to  you,  and  what  are  their  phone  numbers?" 
You  can  see  where  I  was  going.  When  I  called  the  direct  reports  and  asked 
them  the  same  question,  none  of  them  articulated  the  same  purpose  as 
the  colonel  and  none  of  them  aligned  with  each  other.  It  seems  leadership 
101  is  easy  to  identify  but  hard  to  implement. 


Reynolds  is  a  faculty  member  at  the  Defense  Acquisition  University's  Executive  Programs, 
Fort  Belvoir,  Virginia.  He  previously  held  several  positions  with  the  U.S.  Coast  Guard,  includ¬ 
ing  director  of  logistics,  deputy  chief  information  officer,  and  research  and  development 
program  director. 
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In  addition  to  having  clear 
and  rational  goals,  leaders 
must  provide  goals  that  touch 
emotional  needs.  To  do  so, 

yuu  uu  nut  use  udtd. 

You  use  stories 
and  pictures. 


That  experience  made  me  realize  that  we  had  been  assuming 
a  level  of  leadership  self-awareness  that  did  not  exist  in  our 
students.  They  were  successful,  and  their  success  was  directly 
related  to  their  leadership  qualities.  But  in  many  cases  they 
couldn't  connect  their  actions  to  leadership  traits  they  pos¬ 
sessed.  This  was  very  disconcerting.  Without  understanding 
their  leadership  strengths,  how  could  they  effectively  mentor 
their  subordinates?  To  encourage  self-awareness,  I  started  a 
program  in  which  students  in  small  groups  define  the  char¬ 
acteristics  of  extraordinary  teams.  This  often  takes  several 
sessions.  After  they  have  defined  an  extraordinary  team,  I 
then  ask,  "What  roles  does  a  leader  play  on  those  teams?" 
I've  done  this  many  times  with  many  different  groups.  While 
their  answers  vary,  the  following  points  have  consistently  been 
identified  as  the  traits  a  leader  must  possess  to  lead  an  ex¬ 
traordinary  team.  I'm  calling  them  Leadership  101. 

The  first  role  a  leader  must  fill  is  to  set  direction  and  es¬ 
tablish  goals.  But  my  colonel  friend  thought  he  had  done 


this.  So  the  deeper  question  is  how  to  set  a  direction  and 
establish  goals  that  lead  to  extraordinary  teams.  Our  learn¬ 
ers  have  come  to  agree  on  two  points  that  must  be  present. 
The  first  point  is  that  the  leader  must  know  and  define  the 
"Why"  of  the  organization.  As  an  executive  coach,  I  was 
working  with  the  PM  of  the  F-22  Raptor  program.  This  pro¬ 
gram  had  existed  for  years.  Yet  I  discovered  in  the  program's 
information  briefs  it  was  common  to  list  the  features  of  the 
aircraft— advanced  avionics,  stealth  and  super  aerodynam¬ 
ics.  The  question  leadership  must  be  able  to  answer  and 
articulate  is  why  we  need  those  features— the  benefits,  if 
you  will.  Or  said  another  way,  what  defense  problems  does 
the  F-22  program  solve?  We  created  two  PowerPoint  slides 
that  did  just  that  (Figures  1  and  2). 

The  leader's  messaging  on  direction  and  goals  also  must  ad¬ 
dress  both  the  rational  and  emotional  elements  within  people. 
People  are  not  like  Spock  from  the  Starship  Enterprise.  We  are 
emotional.  So  in  addition  to  having  clear  and  rational  goals, 
leaders  must  provide  goals  that  touch  emotional  needs.  To 
do  so,  you  do  not  use  data.  You  use  stories  and  pictures.  One 
Army  officer  in  charge  of  installing  computer  cabling  on  Army 
bases  used  the  PowerPoint  slides  in  Figures  3  and  4  to  high¬ 
light  that  his  organization  was  responsible  for  connecting  the 
Army  to  the  world.  When  showing  the  slides,  he  told  engaging 
stories  about  how  the  connectivity  they  provided  saved  lives  in 
battle  and  let  soldiers  overseas  communicate  with  family  back 
home.  He  closed  by  listing  several  tangible  and  very  rational 
goals  for  the  next  year.  He  touched  the  emotional  and  rational 
buttons  of  his  team. 

Manifestation  of  Culture 

The  second  role  of  a  leader  is  to  set  the  team's  culture.  Cul¬ 
ture  is  the  way  a  team  behaves  based  on  the  values,  ritu¬ 
als,  heroes  and  symbols  they  accept  and  which  are  passed 
along  by  imitation  from  one  generation  to  the  next.  Values 
tend  to  range  along  continuums— for  example,  transparency, 
secrecy;  collaboration,  isolation;  empowerment,  microman¬ 
agement,  etc. 


Figure  1.  Raptor’s  Unique  Attributes 
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Figure  2.  The  Synergy  of  Stealth  and  Super 
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Figure  3.  Connecting  the  Army .  .  . 


Our  Simplified  Mission 


Many  values  remain  unconscious  and  unwritten  to  those  who 
hold  them.  Therefore,  those  values  often  cannot  be  discussed 
nor  directly  observed  by  others.  Values  can  only  be  inferred 
from  how  people  behave  under  different  circumstances.  Sym¬ 
bols,  heroes  and  rituals  are  the  tangible  or  visual  aspects  of  the 
practices  of  a  culture.  The  true  cultural  meaning  of  the  prac¬ 
tices  is  intangible:  This  is  revealed  only  when  the  practices  are 
interpreted  by  the  insiders.  If  a  team's  direction  is  set  through 
a  combination  of  rational  and  emotional  arguments,  setting 
culture  is  an  emotional  pull. 

Hands  down,  I  believe  the  military  Service  that  has  done  the 
best  job  of  setting  its  culture  is  the  Marine  Corps.  All  four 
Services  attend  our  program,  but  I've  found  the  only  Service 
whose  members  can  articulate  their  culture— values,  rituals, 
symbols  and  heroes— is  the  Marine  Corps.  People  in  other 
Services  struggle  to  explain  their  cultural  norms,  but  ironically 
they  usually  can  define  the  Marine  Corps  culture.  How  have 
the  Marines  achieved  this?  First,  they  have  defining  slogans: 
(1)  Every  Marine  is  a  rifleman,  (2)  officers  eat  last,  (3)  with  the 
sun  comes  the  guns,  and  (4)  the  Few,  the  Proud,  the  Marines. 
These  have  not  changed  over  time.  Second,  their  rituals  and 
rewards  reinforce  the  slogans.  For  example,  during  their  an¬ 
nual  "birthday"  celebration,  Marines  will  travel  many  miles  to 
attend  a  ceremonial  cake  cutting  in  which  the  oldest  Marine 
in  the  room  will  serve  the  youngest  and  the  Marine  Corps 
commandant,  via  video  messaging,  will  remind  everyone  of 
the  proud  traditions  of  Marines.  There  are  no  ex-Marines!  Ex¬ 
traordinary  teams  have  leaders  that  don't  let  culture  happen 
by  itself.  They  drive  the  culture  needed  for  the  team  to  excel. 

The  third  role  of  a  leader  is  to  provide  a  winning  environment. 
When  discussing  this  role,  our  learners'  initial  focus  often  is 
on  resources.  That  is,  does  the  team  have  the  right  people, 
enough  money,  workable  schedule,  the  proper  materials  and 
access  to  information?  In  times  of  austere  budgets,  studies 
have  suggested  leaders  often  take  on  too  much  risk  in  sched¬ 
ule,  performance  and  cost  commitments.  So  teams  every  day 
arrive  to  work  with  data  suggesting  they  are  failing  in  their 


primary  duties.  In  many  of  those  instances,  the  leaders  failed 
their  teams. 

As  our  students  progress  into  a  deeper  analysis  of  what  it 
means  to  provide  a  winning  environment,  they  feel  the  leader 
must  build  spheres  of  influence  that  allow  him  or  her  to  have 
sway  with  key  stakeholders  and  their  own  chain  of  command. 
In  "Empowering  Yourself,"  Harvey  Coleman  suggests  the  for¬ 
mula  for  successful  leaders  is  PIE— Performance,  Image  and 
Exposure.  They  must  perform  exceptionally  well,  cultivate 
the  proper  image  and  manage  exposure  so  that  the  right 
people  will  know  them.  It  is  important  to  note  that  leaders 
are  not  doing  this  for  personal  satisfaction  or  gain,  but  to 
provide  a  winning  environment  for  their  teams  by  having  the 
influence  needed  to  get  things  done.  The  author  weighted  the 
three  elements  as  follows:  Performance,  10  percent;  Image, 
30  percent;  and  Exposure,  60  percent.  These  interesting 
ratios  suggest  that  having  a  good  reputation  is  only  part  of 
the  equation;  one  must  make  sure  there  is  a  wide  exposure 
of  that  reputation. 

In  addition,  our  students  believe  a  leader  must  create  an  en¬ 
vironment  for  the  team  to  evaluate  and  learn  as  it  performs 
(some  students  listed  this  trait  under  culture).  In  the  book 
"Clear  Leadership,"  Gervase  Bushe  says  the  rapid  rate  of 
change  in  the  world  today  often  means  our  internal  mental 
models  for  addressing  challenges  no  longer  apply.  I  believe 
this  is  what  Under  Secretary  of  Defense  for  Acquisition,  Tech¬ 
nology,  and  Logistics  Frank  Kendall  means  when  he  asks  us 
to  "think."  He  is  asking  us  to  design  our  way  forward.  To  de¬ 
sign  your  way  forward,  your  team  must  constantly  work  to 
improve.  After  meetings,  reviews  and  decisions,  a  leader  must 
hold  timely  discussions  on  what  worked  and  on  what  could 
be  improved  in  the  next  meeting,  review  or  decision  process. 
You  are  not  assigned  as  a  leader  to  maintain  the  status  quo. 

The  fourth  role  of  a  leader  is  to  provide  mentorship— that 
is,  to  grow  the  team.  The  last  paragraph  was  about  improv¬ 
ing  the  synergy  amongst  team  members.  This  is  about  the 


Figure  4.  ...  to  the  World 
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As  for  setting  standards  of 
performance,  our  students 
feel  “one  gets  what  one 
accepts."  Leaders  must  set 
and  enforce  standards 
of  performance 
that  drive 

results.  t 


performance  of  individual  members  of  the  team.  All  leaders 
claim  to  care  about  their  teams.  What  separates  the  lead¬ 
ers  of  extraordinary  teams  is  that  they  go  beyond  saying  they 
care  to  developing  plans  and  strategies  for  demonstrating  their 
concern  for  their  employees'  growth  and  well-being.  It  is  on 
their  calendar.  For  example,  one  of  my  coaching  clients  built 
people  growth  into  his  strategy  and  measurement  system.  He 
received  quarterly  reports  that  measured:  (1)  promotions,  both 
internal  and  external,  (2)  training  hours,  (3)  overtime  hours, 
and  (4)  awards  related  to  improved  performance.  Discussions 
then  would  be  held  to  capture  what  drove  good  numbers  and 
which  areas  needed  improvement.  This  created  a  culture  in 
which  all  members  of  the  senior  leadership  team  saw  growing 
the  next  generation  of  leaders  as  part  of  their  primary  duty. 

The  fifth  and  final  role  of  a  leader  as  defined  by  the  students  is 
to  set  the  standards  of  performance  and  boundary  conditions. 
Our  learners  often  struggle  in  their  talks  on  what  makes  good 
and  bad  boundary  conditions.  Let  me  explain  using  a  ship¬ 
ping  lane  metaphor  to  describe  how  bad  boundaries  can  limit 
team  success.  Those  who  have  been  to  sea  on  a  military  vessel 
are  familiar  with  the  Captain's  Standing  Orders.  These  orders 
set  the  boundary  conditions  for  when  the  captain  should  be 
informed  of  shipboard  conditions.  They  usually  include  com¬ 
ments  on  how  close  another  vessel  can  approach  or  when  a 
weather  condition  change  requires  the  captain's  notification. 

Picture  the  entrance  to  any  major  port  of  the  United  States. 
We  have  shipping  lanes,  and  some  distance  from  the  shipping 
lanes  there  is  shoal  water.  Navy  doctrine  allows  Navy  ships  to 
enter  ports  outside  the  shipping  lanes.  To  reduce  the  risk  of  a 
mishap,  the  Navy  provides  doctrine  on  the  size  of  the  buffer 
the  Navy  vessels  must  give  both  the  shipping  lanes  and  the 
shoals.  In  a  way,  they  have  provided  a  box  in  which  the  vessels 
can  operate.  Vessel  Commanding  Officers  (COs),  not  wanting 


to  get  in  trouble,  write  their  Standing  Orders  so  that  the  buf¬ 
fer  on  each  side  is  larger— to  make  sure  they  stay  in  the  box. 
This  means  the  box  the  Navy  ship  can  operate  in  is  now  much 
smaller.  The  Officer  Of  the  Deck  (OOD),  not  wanting  to  get  in 
trouble  with  the  CO,  issues  orders  to  the  helmsman  increasing 
the  buffer  given  to  both  the  shoals  and  shipping  lanes.  The  box 
for  the  ship  to  operate  in  has  once  again  been  reduced. 

The  poor  helmsman,  not  wanting  to  get  in  trouble  with  the 
OOD,  stays  on  a  straight  line.  There  is  no  room  for  movement. 
Soon  these  boundaries  are  not  considered  discretionary  but 
are  required  operating  procedures.  My  students  have  con¬ 
cluded  that  the  key  for  successful  boundary  condition  setting 
is  for  leaders  to  adjust  the  boundaries  to  match  the  talents  of 
their  people,  not  to  constrict  them.  The  boundaries  should 
change  as  the  people  change  and  grow. 

As  for  setting  standards  of  performance,  our  students  feel 
"one  gets  what  one  accepts."  Leaders  must  set  and  enforce 
standards  of  performance  that  drive  results.  One  of  my  first 
coaching  clients  stressed  his  team  should  deliver  what  it  prom¬ 
ises  across  the  spectrum  of  cost,  schedule  and  performance. 
One  of  his  commanders  made  bold  projections  about  his  prod¬ 
uct  and  stuck  with  them  even  after  careful  and  detailed  ques¬ 
tioning.  My  client  then  created  a  large  poster  that  highlighted 
the  next  scheduled  milestone  on  the  commander's  program. 
As  the  date  approached,  the  commander  came  forward  and 
admitted  he  couldn't  deliver  on  his  promise.  In  response,  my 
client  held  an  all-hands  meeting  and  made  the  failure  public 
and  stressed  how  this  example  counters  all  this  team  stood  for 
as  the  warfighter's  trusted  provider  of  capability.  The  culture 
of  that  organization  changed  that  day  as  new  heroes  were 
identified  and  new  stories  described  the  team.  My  client  ex¬ 
pected  excellence,  and,  in  holding  his  team  to  that  standard,  he 
received  it,  and  that  excellence  became  a  key  element  of  team 
pride.  The  team  members  saw  their  standards  of  performance 
as  differentiators  from  other  teams. 

In  summary,  we've  asked  the  senior  executives  chosen  to  lead 
DoD's  most  difficult  acquisition  challenges  to  reflect  on  the 
roles  leaders  must  undertake  to  build  extraordinary  teams. 
Their  consensus  opinion  is  that  a  leader  sets  a  direction  and 
makes  sure  the  team  knows  why.  A  leader  sets  team  values, 
rituals  and  symbols  and  identifies  and  rewards  those  heroes 
modeling  them.  A  leader  provides  a  winning  environment  in 
which  the  resources  are  right,  external  influences  are  con¬ 
trolled  and  teams  constantly  grow  and  improve.  A  leader  cares 
about  and  grows  each  team  member.  Finally,  a  leader  sets  and 
enforces  standards  of  performance. 

My  experience  is  that  few  leaders  reach  the  101  level  of  perfor¬ 
mance.  They  let  the  tyranny  of  today's  challenges  distract  their 
focus  on  fulfilling  their  leadership  responsibilities.  I  challenge 
you  to  review  your  calendar  and  see  how  much  of  your  time 
is  committed  to  Leadership  101.  & 


The  author  can  be  contacted  at  scott.reynolds@dau.mil. 


Defense  AT&L:  November- December  2015 


20 


Brian  Schultz  ■  David  Dotson 


“Price  is  what  you  pay. 
Value  is  what  you  get/ 
—Warren  Buffett 


Competition  in  acquisition  is  an  important  topic  and  has  been  since  the  Department  of 
Defense  (DoD)  started  acquiring  systems  from  the  defense  industry.  The  key  premise 
is  that  DoD  will  get  greater  value  for  the  price  paid  as  a  result  of  competition.  Some 
studies  suggest  savings  in  the  15  percent  to  25  percent  range  and  even  greater  under 
some  conditions  as  a  result  of  competition.  However,  greater  value  is  not  always  tied  to 
lower  prices  or  cost  savings.  Greater  value  can  be  realized  through  a  superior  technical  solution 
as  part  of  a  trade-off  of  price  and  other  factors  in  a  source  selection. 

This  article  addresses  our  thoughts  in  the  best  value  discussion,  including  the  use  of  the  Lowest  Price  Technically 
Acceptable  (LPTA)  method.  According  to  the  Government  Accountability  Office's  Report  14-884  ( Factors  DOD 
Considers  When  Choosing  Best  Value  Processes  Are  Consistent  with  Guidance  for  Selected  Acquisitions)  dated  July  2014, 
use  of  LPTA  on  contracts  valued  at  more  than  $25  million  has  increased  10  percent  while  full  trade-offs  decreased 
by  9  percent  over  the  last  few  years.  This  increased  use  of  LPTA  has  generated  a  lot  of  discussion  and  interest 
from  industry.  The  following  is  just  a  small  sample  of  some  recent  news  articles  and  blog  titles  that  discuss  use  of 
LPTAs  versus  the  trade-off  for  best  value  source  selections: 


Schultz  is  a  professor  of  Program  Management,  at  the  Defense  Acquisition  University  (DAU)  Capital  Northeast  Region,  Fort  Belvoir,  Virginia. 
Dotson  is  a  professor  of  Contract  Management  at  the  DAU  Mid-Atlantic  Region  in  California,  Maryland. 
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■  "Five  Factors  Plaguing  Pentagon  Procurement,"  American 
Enterprise  Institute,  Dec.  13,  2013. 

■  "Executive  Slams  'Lowest  Price,  Technically  Acceptable' 
Acquisition  Regimes"  National  Defense  Magazine ,  Jan.  16, 
2014. 

■  "Eliminate  LPTA  and  staff  augmentation  contracting 
other  than  for  commodity  services  with  minimal  mission 
impact,"  Defense  Business  Board  Report ,  July  24,  2014. 

■  "LPTA  Contracts  Stifle  Government  Innovation,"  Informa- 
tionWeek,  Oct.  25,  2013. 

■  "Is  the  Government  Starting  to  hate  LPTA  too?"  Washing¬ 
ton  Technology ,  June  7,  2013. 

■  "Too  Much  Competition  can  Reduce  Incentives  for  Inno¬ 
vation,"  Lexington  Institute,  April  22,  2013. 

■  "Best  Value  or  LPTA?  One  Size  Does  Not  Fit  All  in  Acqui¬ 
sition,"  lntegrityMatters.com,  Feb.  27,  2013. 

■  "Pentagon  launching  new  review  of  LPTA  contracts," 
Federalnewsradio.com,  Dec.  8,  2014. 

The  March-April  2015  issue  of  Defense  AT&L  magazine  in¬ 
cluded  articles  on  the  use  of  LPTA,  one  ("Getting  'Best  Value' 
for  the  Warfighter  and  the  Taxpayer")  by  Frank  Kendall,  Under 
Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics 
and  another  article  providing  an  industry  perspective,  ("Low¬ 
est  Price  Technically  Acceptable  Overrated,  Overused?")  by 
Will  Goodman.  Both  articles  highlighted  the  concern  that  LPTA 
should  be  used  only  in  limited  cases.  But  they  differed  on  the 
approach  that  should  be  used  in  assessing  the  suitability  of  an 
LPTA  source  selection  (value  gained  from  performance  above 
an  established  minimum  versus  assessing  the  long-term  im¬ 
pacts  on  capability  areas  and  changing  the  Federal  Acquisition 
Regulation  [FAR]). 

Goodman  suggested  that  looking  at  the  acquisition  as  a 
single  transaction  is  deficient.  He  suggested  that  contract¬ 
ing  officers  should  be  directed  to  assess  capability  area 
outcomes  over  the  course  of  multiple  solicitations  in  de¬ 
termining  whether  LPTA  should  be  used.  "To  address  the 
deficiency,  the  FAR  should  point  out  that  LPTA  is  ideal  for 
commodities  and  commoditized  services  and  note  that  con¬ 
tracting  officers  should  consider  the  possible  and  probable 
long-term  impacts  of  an  LPTA  source  selection  on  the  ca¬ 
pability  area  addressed  by  the  solicitation." 

While  it  is  an  innovative  idea,  we  don't  believe  the  FAR  sug¬ 
gestion  above  is  practical.  Program  level  contracting  offi¬ 
cers  are  not  typically  in  a  position  to  do  this  and  should  not 
be  expected  to  make  this  kind  of  portfolio-wide  assessment. 
Furthermore,  it  would  be  difficult,  even  for  senior  leaders 
who  oversee  large  portfolios  of  programs,  to  tie  one  or  even 
multiple  actions  to  a  future  mission  area  or  mission  capa¬ 
bility  outcome.  While  forecasting  and  planning  for  future 
capability  needs  is  an  important  and  ongoing  effort,  it  could 
be  very  difficult  to  determine  long-term  impacts  based  on 
the  use  of  a  source-selection  method  on  any  given  contract. 
It  would  also  raise  questions  about  whether  DoD  is  unnec¬ 
essarily  paying  more  for  a  product  or  service  because  of 


future  outcomes  that  are  difficult  to  forecast  and  perhaps 
even  more  difficult  to  measure. 

Complexity  and  Risk  Framework 

We  suggest  an  alternative  approach  to  choosing  a  source- 
selection  method  based  on  a  complexity  and  risk  framework 
intended  for  use  by  the  government  acquisition  team.  It  would 
involve  an  integrated,  multifunctional  effort  in  close  coordina¬ 
tion  with  the  contracting  officer.  This  assessment  would  go 
beyond  just  looking  at  performance  requirements,  to  assessing 
the  following  programmatic  areas: 

■  Organizational  landscape 

■  Mission  and  operational  interfaces 

■  Industrial  capabilities 

■  Deliverables  or  outcomes 

■  Risk  and  opportunity  management 

We  would  argue  that  this  level  and  depth  of  assessment  will 
provide  a  more  stable  foundation  for  making  informed  deci¬ 
sions  about  source-selection  strategies  through  the  life  of  the 
acquisition  program. 

The  proposed  framework  is  based  on  our  past  experiences  in 
pre-Request  for  Proposal  planning  efforts  and  is  consistent 
with  the  current  statutory,  regulatory  and  policy  guidance. 
Note  that  while  risk  and  risk  management  are  well  defined 
in  DoD,  this  is  not  the  case  for  complexity.  The  FAR  and  DoD 
guidance  are  clear  that  both  complexity  and  risk  of  the  re¬ 
quirements  are  relevant  in  assessing  the  relative  importance 
of  price  and  nonprice  factors  (e.g.,  technical)  and  value  above 
a  minimum  performance  level.  This  framework  can  help  DoD 
teams  assess  both  complexity  and  risk. 

The  following  is  a  brief  summary  of  each  assessment  area: 

Organizational  Landscape:  The  overall  health  and  stability 
of  an  acquisition  organization  can  have  significant  implica¬ 
tions  for  its  ability  to  plan,  develop  and  execute  acquisition 
efforts.  Organizational  issues  can  directly  affect  the  abil¬ 
ity  to  oversee  and  manage  certain  efforts  (e.g.,  complex 
trade-off  decisions),  requiring  greater  contractor  technical 
competence  and  increasing  overall  risk  to  the  program.  For 
example,  less  stable  organizations  with  management  chal¬ 
lenges  may  need  additional  time  and  assistance  to  develop 
a  performance  work  statement  in  response  to  a  services 
acquisition  requirement. 

Such  an  unstable  acquisition  organization  may  be  better 
served  with  more  senior-level  services  support  that  can  react 
quickly  to  changes  in  the  environment  and  deliver  advisory 
services  with  less  oversight.  Use  of  an  LPTA  in  this  example 
may  not  result  in  a  good  outcome  as  organizational  risk  and 
complexity  are  high. 

Sample  questions  that  should  be  considered  for  organizational 
landscape  include:  Is  the  program  office  staffed  adequately? 
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"The  bitterness  of  poor 
quality  remains  long  after 
the  sweetness  of  low  price  is 
forgotten." 
—Benjamin  Franklin 


Is  there  high  turnover  in  key  personnel?  Is  there  strong  budget 
and  advocacy  support  from  stakeholders  and/or  sponsors?  Is 
there  strong  alignment  with  the  agency  strategic  plan?  What 
is  the  availability  of  key  organic  resources  to  plan  and  manage 
the  contractual  efforts,  both  pre-  and  post-award?  Has  the 
organization  accomplished  its  goals  effectively? 

Mission  and  Operational  Interfaces:  The  operational  environ¬ 
ment  of  the  system  or  organization  can  result  in  added  risks 
and  complexities.  For  example,  some  systems  must  integrate 
with  complex  communications  and  information  networks  that 
involve  specialized  technical  expertise  in  order  to  be  effective. 
These  operational  environments  can  rapidly  change  as  new 
threats  emerge  and  new  technologies  are  introduced. 

For  example,  we  both  worked  on  airborne  command  and 
control  system  programs  that  had  to  integrate  into  multiple 
networks  and  required  interoperability  with  joint  and  coalition 
partners,  some  with  very  different  system  configurations  and 
exchange  requirements.  This  added  great  complexity  to  the 
program  in  order  to  support  current  operations  and  planned 
future  capability  upgrades. 

Program  teams  should  address  how  this  system  or  program 
or  service  fits  into  the  bigger  picture.  What  other  system  in¬ 
terfaces,  networks  and  information  exchange  requirements 
are  needed?  How  well  understood  and  stable  are  the  require¬ 
ments?  What  is  the  operational  or  organizational  urgency? 
What  kinds  of  emerging  threats  or  new  contingency  opera¬ 
tions  are  relevant? 

Industrial  Capabilities:  The  ability  and  track  record  of  industry 
(including  suppliers)  in  the  relevant  domains  should  be  as¬ 
sessed  as  early  as  possible.  Previous  execution  issues,  poor 
incumbent  performance,  poor  financial  performance,  and  lack 


of  qualified  suppliers  are  examples  of  indicators  for  possible 
high  complexity  and  risk.  Market  research  plays  an  important 
role  in  gathering  information  needed  for  this  assessment. 

For  example,  consider  a  complex  air  traffic  control  radar  sys¬ 
tem  that  has  matured  to  the  point  where  it  is  considered  a 
production  commodity.  This  system  capability  has  multiple 
companies  whose  product  meets  the  mandated  requirements. 
While  the  system  design  is  fairly  complex,  the  industrial  capa¬ 
bility  could  be  rated  as  having  low  complexity  and  risk  due  to 
the  nature  of  industry's  ability  to  deliver  this  product  and  the 
track  record  of  previous  contracts. 

Sample  questions  to  be  considered  include:  What  capabili¬ 
ties  does  industry  offer  to  meet  the  stated  needs?  What  is 
the  past  performance  of  previous  contracts  for  similar  ef¬ 
forts?  What  is  the  industry  confidence  level  in  estimating  its 
costs  to  perform  the  work?  What  is  the  financial  health  and 
commitment  to  cover  any  overruns  of  interested  companies? 
How  much  unique  domain  knowledge  of  the  agency  chal¬ 
lenges  is  necessary  to  execute  the  contract?  How  will  this 
knowledge  be  obtained?  Is  it  the  contractor's  responsibility 
to  have  in-house  expertise  or  will  the  government  provide  the 
support  needed  to  ensure  adequate  knowledge?  Note  that,  in 
some  cases,  the  government  team  may  need  to  dedicate  staff 
to  guide  contractor  efforts  to  ensure  the  product  or  service 
is  tailored  appropriately  to  meet  the  agency  need.  A  good 
example  of  this  is  configuring  software  to  meet  a  unique  or¬ 
ganizational  requirement. 

Deliverables  and  Outcomes:  The  product  or  service  outcome, 
clarity  and  scope  of  requirements,  and  the  amount  of  develop¬ 
ment  involved  should  be  assessed  as  well  as  the  determination 
of  the  value  of  (and  what  we  are  willing  to  pay  for)  increased 
levels  of  technical  performance. 

Several  questions  should  come  to  mind  in  assessing  the  deliv¬ 
erables  and  expected  outcomes  of  the  acquisition:  What  is  the 
expected  outcome  and  how  is  success  defined?  What  prod¬ 
ucts  and  deliverables  are  required  and  what  are  the  accep¬ 
tance  criteria  or  acceptable  quality  standards  for  them?  How 
much  new  development  work  is  required?  Is  the  product  a 
commercial  item  (or  commercial  service  offering)?  If  a  legacy 
effort,  what  issues  and  challenges  were  encountered  in  previ¬ 
ous  contracts  and  how  difficult  were  they  to  overcome?  What 
is  the  value  of  performance  above  the  minimum  threshold? 
This  is  a  question  recently  posed  by  Kendall.  If  a  services  ac¬ 
quisition,  what  skill  levels  and  expertise  are  anticipated?  What 
domain  knowledge  of  the  product  or  mission  area  is  required? 

Risk  and  Opportunity  Management:  After  assessing  the 
above,  teams  should  identify  risks  to  successful  contract 
performance.  Starting  the  analysis  with  the  previous  areas  is 
recommended  because  doing  so  can  assist  in  identifying 
risk  areas  and  their  probability  and  consequences.  It  is  im¬ 
portant  to  continuously  manage  risks— meaning  actions 
will  be  taken  to  mitigate  risks,  and  the  organization  will 
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Figure  1.  Integrated  Assessment  Rating  Scale 
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^Multiple  approaches  could  be  appropriate  with  moderate  levels  of  risk  and  complexity 
LPTA:  Lowest  Price  Technically  Acceptable 
CA:  Combination  Approach 
TO:  Trade-off 

Assessment  scale  not  absolute— tailor  to  program  circumstances 


As  shown  in  Figure  1,  high  com¬ 
plexity  and  risk  suggest  that  a 
trade-off  source  selection  should 
be  considered.  Moderate  com¬ 
plexity  and  risk  suggest  that  ei¬ 
ther  a  trade-off  or  a  combination 
approach  could  be  considered. 
Finally,  low  complexity  and  risk 
align  with  the  LPTA  approach. 
This  proposed,  notional  scale  also 
maps  to  the  best  value  continuum 
associated  with  the  relative  im¬ 
portance  of  price  and  nonprice 
factors  and  incorporates  the  re¬ 
sults  of  both  complexity  and  risk 
into  the  assessment. 


implement  a  process  to  govern  the  risk  management  ef¬ 
forts  throughout  the  contract.  This  often  occurs  as  part  of 
a  risk  management  review  board  that  meets  periodically  to 
review  risk  management  efforts.  Opportunity  management 
should  be  assessed  for  initiatives  that  can  reduce  costs  and 
add  greater  value  to  the  effort  within  the  scope  of  the  exist¬ 
ing  contract. 

The  culmination  of  this  effort  results  in  an  integrated  assess¬ 
ment  of  low,  moderate  or  high  complexity  and  risk  based  on 
the  answers  to  relevant  questions  in  each  area.  A  roll-up  of  the 
areas  to  one  color  rating  is  a  judgment  call  based  on  the  rela¬ 
tive  importance  of  the  different  areas  that  are  relevant  for  the 
program.  The  notional  rating  scale  shows  green  as  low,  yellow 
as  moderate,  and  red  as  high  complexity  and  risk. 


Closing  Thoughts 

As  Franklin  noted  centuries  ago,  low  price  is  not  always  the 
best  deal,  especially  if  either  the  quality  or  outcome  of  our 
acquisition  is  at  stake.  Source  selection  and  use  of  an  appropri¬ 
ate  method  to  achieve  best  value  is  an  important  decision  that 
should  involve  a  deliberate  process  to  ensure  we  have  thought 
through  what  we  need  and  how  best  to  get  it.  A  framework 
for  assessing  program  complexity  and  risk  may  prove  useful 
for  some  in  making  an  informed  decision  about  the  source- 
selection  method.  The  real  value  of  using  an  assessment  meth¬ 
odology  similar  to  this  is  not  an  absolute  answer  but  rather  the 
critical  thinking  that  supports  good  acquisition  outcomes,  both 
for  DoD  and  industry— and  ultimately  for  the  taxpayer.  & 

The  authors  can  be  contacted  at  brian.schultz@dau.mil  and 
david.dotson@dau.mil. 


MDAP/MAIS  Program  Manager  Changes 


With  the  assistance  of  the  Office  of  the  Secretary  of  De¬ 
fense,  Defense  AT&L  magazine  publishes  the  names  of  in¬ 
coming  and  outgoing  program  managers  for  major  defense 
acquisition  programs  (MDAPs)  and  major  automated  infor¬ 
mation  system  (MAIS)  programs.  This  announcement  lists 
all  such  recent  changes  of  leadership,  for  both  civilian  and 
military  program  managers. 

Army 

COL  James  C.  Mills  relieved  COL  Gary  D.  Stephens  as 

project  manager  for  the  Precision  Fires  Rocket  and  Missile 
Systems  (PFRMS)  on  July  15. 

COL  William  D.  Jackson  relieved  COL  Thomas  H.  Todd  as 

project  manager  for  Utility  Helicopters  (UH)  on  July  15. 

COL  Shane  N.  Fullmer  relieved  COL  John  Cavedo,  Jr.  as 

project  manager  for  Joint  Program  Office,  Joint  Light  Tactical 
Vehicles  (JPO  JLTV)  on  July  31. 


COL  Gregory  H.  Coile  relieved  COL  Edward  J.  Swanson  as 

project  manager  for  the  Warfighter  Information  Network- 
Tactical  (WIN-T)  on  July  15. 

COL  Donald  W  Hurst,  III  relieved  COL  Sandra  L.  Vann- 
Olejasz  as  project  manager  for  DoD  Biometrics  on  July  16. 

Air  Force 

Col.  Michael  Harm  relieved  Col.  Scott  Owens  as  program 
manager  for  the  Theater  Battle  Control  Systems  program 
on  Aug.  3. 

Col.  Darien  Hammett  relieved  Col.  Carlin  Heimann  as  pro¬ 
gram  manager  for  the  RQ-4  Global  Hawk  System  program 
on  July  5. 

Col.  Steven  Whitney  relieved  Brig.  Gen.  William  Cooley 

as  program  manager  for  the  Global  Positioning  System  Pro¬ 
gram  on  July  8. 
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A  Contract  Requirement 
Rule  for  Program  Managers 

Paul  Solomon 


The  National  Defense  Authorization  Act  (NDAA)  for  Fiscal  Year 
(FY)  2016  (NDAA)  includes  many  acquisition  reforms.  The  Senate 
version  included  a  provision  that  would  have  required  the  Secretary 
of  Defense  to  develop  standards,  policies  and  guidelines  based  on 
nationally  accredited  standards  for  program  and  project  manage¬ 
ment  as  well  as  policies  to  monitor  compliance.  The  Project  Management  Body 
of  Knowledge  (PMBOK  Guide )  is  the  only  qualifying  standard.  The  provision 
was  not  adopted  during  conference  with  the  House.  However,  if  the  PMBOK 
Guide  were  made  a  contract  requirement  to  replace  the  Earned  Value  Man¬ 
agement  System  (EVMS)  standard  (ANSI-748),  any  program  manager  (PM) 
finally  would  be  able  to  identify  and  pinpoint  emerging  problems  on  a  timely 
basis  and  act  as  early  as  possible  to  resolve  problems.  This  article  discusses 
the  content  and  benefits  of  the  PMBOK  Guide  but  also  includes  guidance 
for  integrating  systems  engineering  (SE)  and  risk  management  with  EVM 
independently  of  the  PMBOK  Guide. 

PM  Responsibilities,  Needs  and  Tools 

Per  Department  of  Defense  Instruction  (DoDI)  5000.02,  Operation  of  the  Defense  Acqui¬ 
sition  System,  PM  responsibilities  include  achieving  the  cost,  schedule  and  performance 
parameters  specified  in  the  Milestone  Decision  Authority-approved  Acquisition  Program 
Baseline  (APB).  Per  the  Defense  Acquisition  Guidebook  (DAG),  the  PM  should  require  con¬ 
tractors  and  government  activities  to  use  internal  management  control  systems  that 
"properly  relate  cost,  schedule,  and  technical  accomplishment."  Also,  per  DAG,  "risk 
management  is  most  effective  when  fully  integrated  with  the  program's  SE  and  manage¬ 
ment  processes." 

During  the  Engineering  and  Manufacturing  Development  (EMD)  phase,  the  PM  must 
develop,  build  and  test  a  product  to  verify  that  all  operational  and  derived  requirements 


Solomon  is  retired  from  Northrop.Grumman  Corp.  He  is  a  Project  Management  Professional  and  co-author  of 
ANSI-748  and  the  book  "Performance-Based  Earned  Value,"  published  in  2007.  Solomon  is  a  1998  recipient 
of  the  Defense  Department's  David  Packard  Excellence  in  Acquisition  Award.  The  author  maintains  a  project 
management  Website,  www.pb-ev.com,  and  welcomes  comments  and  questions. 
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have  been  met.  The  artifacts  and  tools  that  a  PM  needs,  per 
DoDI  5000.02,  DAG ,  and  the  SE  Plan  Guide  (SEP),  include: 

■  Capability  requirements 
•  SEP 

■  Functional  and  physical  characteristics  of  the  system 
design  integrated  with  the  SEP 

■  Technical  baseline 

■  Product  baseline  for  all  configuration  items 

■  System  baselines  (functional,  allocated,  product) 

■  Requirements  traceability  between  the  system's  technical 
requirements  and  work  breakdown  structure  (WBS) 

■  Technical  performance  measures  (TPM)  and  metrics  to 
assess  program  progress 

In  2009r  the  DoD  reported  to 
the  House  and  Senate  oversight 
committees  that  the  "utility 
of  EVM  has  declined  to  a  level 
where  it  does  not  serve  its 
intended  purpose." 

■  Risk  management  and  tracking 

■  Earned  Value  Management  (EVM) 

■  Performance  Measurement  Baseline  (PMB) 

Unfortunately,  there  are  omissions  in  the  acquisition  process, 
regulations  and  guides  that  impede  the  PM's  success.  There 
are  no  requirements  for  contractors  to  use  internal  manage¬ 
ment  control  systems  that  properly  relate  cost,  schedule  and 
technical  accomplishment  with  the  following  tools  and  arti¬ 
facts:  SEP;  Requirements  traceability  between  the  system's 
technical  requirements  and  WBS;  system  baselines  (func¬ 
tional,  allocated,  product);  incorporation  of  product  baseline 
into  PMB;  TPMs;  and  risk  management  and  tracking. 

EVM  Shortcomings 

The  use  of  EVM  is  not  sufficient  to  provide  the  PM  with  valid 
information  on  cost,  schedule  and  technical  performance. 
Unfortunately,  EVM,  when  implemented  by  ANSI-748,  is  not 
designed  to  provide  performance  toward  achieving  the  techni¬ 
cal  or  product  baseline. 

First,  ANSI-748  cites  only  the  "work  scope''  not  the  techni¬ 
cal  baseline  or  the  product  scope  that  is  in  the  APB.  Sec¬ 
ond,  ANSI-748  measures  only  the  "quantity  of  work  per¬ 
formed"  and  not  the  quality  of  the  system  being  designed 
and  tested.  Third,  the  use  of  TPMs  in  ANSI-748  is  optional. 
Consequently,  DoDI  5000.02  is  not  convincing  in  its 


assertions  that  EVM  "promotes  an  environment ...  in  which 
problems  are  identified,  pinpointed,  and  acted  upon  as  early  as 
possible"  and  also  that  it  is  a  "powerful  program  planning 
and  management  tool." 

The  title  of  ANSI-748  confirms  that  it  is  only  designed  to  man¬ 
age  a  statement  of  work  and  not  a  project  that  concludes  with 
a  product.  Its  title  states  that  it  is  a  "Guide"  for  "Coordination 
of  Work  Scope,  Schedule,  and  Cost  Objectives."  "Product"  or 
"technical"  objectives  are  absent. 

The  purpose  of  EVM  is  stated  in  Office  of  Management  and 
Budget  (OMB)  Circular  No.  A-11,  Planning ,  Budgeting ,  Acquisi¬ 
tion  and  Management  of  Capital  Assets.  Section  300-5  of  OMB 


Circular  A-11  states  that  performance-based  acquisition  man¬ 
agement  should  be  based  on  the  EVMS  standard  and  measure 
progress  toward  milestones,  cost,  capability  to  meet  specified 
requirements,  timeliness  and  quality. 

However,  in  2009,  the  Department  of  Defense  (DoD)  reported 
to  the  House  and  Senate  oversight  committees  that  the  "util¬ 
ity  of  EVM  has  declined  to  a  level  where  it  does  not  serve  its 
intended  purpose."  Per  the  report,  the  PM  should  ensure  that 
the  EVM  process  measures  the  quality  and  technical  maturity 
of  technical  work  products  instead  of  just  the  quantity  of  work 
performed.  The  report  stated  that  EVM  can  be  an  effective 
program  management  tool  only  if  the  EVM  processes  are  aug¬ 
mented  with  a  rigorous  SE  process  and  SE  products  are  costed 
and  included  in  EVM  tracking. 

Now,  6  years  later,  Congress  still  is  considering  EVM  as  a 
target  for  acquisition  reform.  For  more  information  on  the 
deficiencies  of  ANSI-748,  including  the  "Quality  Gap,"  see 
my  article  in  the  November-December  2010  issue  of  Defense 
AT&L,  "Earned  Value  Management  Acquisition  Reform".  (See 
http://www.pb-ev.com/ndaa-.html  or  http://www.dau.mil/ 
pubscats/ATL%20Docs/Nov-Dec10/Solomon.pdf.) 

A  Project  Management  Standard 

EVM,  based  on  ANSI-748,  is  used  primarily  by  federal  con¬ 
tractors  when  contractually  required.  A  more  powerful  tool  is 
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the  ANSI  standard  that  voluntarily  is  used  worldwide  because 
it  works,  not  because  it  is  imposed  by  federal  acquisition 
regulations.  It  is  the  Project  Management  Institute  (PMI) 
PMBOK  Guide. 

The  needs  of  the  PM  that  are  covered  by  the  PMBOK  Guide 
but  absent  in  ANSI-748  include  technical  or  product  baseline; 
requirements  management  and  traceability;  and  risk  manage¬ 
ment.  The  PMBOK  Guide  contains  many  artifacts  and  tools  that 
have  no  counterpart  in  ANSI-748,  including: 

■  Product  scope  description  documenting  the  characteris¬ 
tics  of  the  product  that  the  project  will  create.  It  progres¬ 
sively  elaborates  the  product's  characteristics  ...  described 
in  the  project  charter  and  requirements  documentation. 

■  Project  scope  involving  the  work  that  needs  to  be  accom¬ 
plished  to  deliver  a  product ...  with  the  specified  features 
and  functions. 

■  Requirements  documentation  provides  the  requirements 
baseline;  it  is  unambiguous  (measurable  and  testable), 
traceable,  complete,  consistent,  and  acceptable  to  key 
stakeholders.  Components  include  functional  require¬ 
ments,  nonfunctional  requirements,  quality  requirements 
and  acceptance  criteria. 

■  Requirements  Management  Plan  includes  product  met¬ 
rics  that  will  be  used. 

■  WBS  Dictionary  includes  quality  requirements,  accep¬ 
tance  criteria. 

■  Scope  Baseline  includes  product  scope  description, 
project  deliverables  and  defines  product  user  acceptance 
criteria. 

■  Control  Scope  or  the  process  of  monitoring  the  status  of 
the  project  and  product  scope  and  managing  changes  to 
the  scope  baseline.  Completion  of  the  product  scope  is 
measured  against  the  product  requirements. 

■  Requirements  Traceability  Matrix  (RTM)  includes  require¬ 
ments  to  project  (including  product)  scope/WBS  objec¬ 
tives,  product  design,  test  strategy  and  test  scenarios. 

■  Conduct  risk  management  planning,  identification,  quali¬ 
tative  risk  analysis,  quantitative  risk  analysis,  response 
planning  and  controlling  risk. 

The  PMBOK  Guide  also  covers  EVM  topics  such  as  scheduling 
(including  network  diagrams),  PMB,  control  accounts,  work 
packages,  earned  value,  variance  analysis,  estimate  at  comple¬ 
tion,  and  management  reserve. 

PMBOK  Guide  Deficiencies 

Some  ANSI-748  guidelines  have  no  equivalent  in  the  PMBOK 
Guide.  These  relate  to  organization  costs,  material  accounting 
and  unit/equivalent/lot  costs.  It  is  recommended  that,  during 
the  acquisition  reform  reviews  of  existing  regulations,  these 
guidelines  be  considered  for  elimination. 

Use  of  TPMs  also  is  optional  in  the  PMBOK  Guide.  Conse¬ 
quently,  any  revision  to  the  acquisition  policies  and  regulations 
should  require  contractors  to  identify  and  use  TPMs. 


Guidance  on  Technical  Performance 


For  practical  guidance  to  implement  the  project  man¬ 
agement  needs  described  above,  with  or  without  the 
PMBOK  Guide ,  see  the  author's  article  in  CrossTalk,  the 
Journal  of  Defense  Software  Engineering ,  "Basing  Earned 
Value  on  Technical  Performance"  (January  2013), 
http://www.pb-ev.com/articles-and-tutorial.html.  The 
article  includes  recommended  contract  language  and 
project  monitoring  techniques  to  ensure  that  contrac¬ 
tors  integrate  technical  performance  and  quality,  includ¬ 
ing  software  functionality,  with  EVM. 

This  Defense  AT&L  article  is  a  sequel  to  previous  ar¬ 
ticles  including  "Integrating  Systems  Engineering  with 
Earned  Value  Management"  in  the  May-June  2004  De¬ 
fense  AT&L  (http://www.dau.mil/pubscats/pubscats/ 
atl/2004_05_06/sol-mj04.pdf)  and  "Path  to  EVM  Ac¬ 
quisition  Reform"  in  the  May-June  2011  issue  of  Defense 
AT&L  (http://www.dau.mil/pubscats/ATL%20Docs/ 
May-Junell/Solomon.pdf).  These  articles  also  are  avail¬ 
able  at  www.pb-ev.com. 

Finally,  if  the  PMBOK  Guide  is  not  utilized,  guidance  for 
integrating  risk  management  with  EVM  is  provided  by 
an  article  in  the  Measurable  News,  "Integrating  Risk  Man¬ 
agement  with  EVM  (Risk  Management  Comes  Out  of 
the  Closet),"  (June  1998,  page  11)  (available  at  http:// 
www.pb-ev.com/articles-and-tutorial.html). 


PM  Success 

Acquisition  reforms  should  include  requirements  for  the  PM 
and  contractors  to  use  PMBOK  Guide  for  EMD  contracts  that 
are  above  specified  threshold  values.  The  PM  finally  will 
have  valid  information  and  tools  needed  to  properly  relate 
cost,  schedule  and  technical  accomplishment;  manage  risk 
and  achieve  the  contract's  cost,  schedule  and  performance 
parameters. 

A  PM  can  ensure  integration  of  technical  performance  with 
EVM  even  if  the  PMBOK  Guide  is  not  utilized.  However,  there 
must  be  a  contractually  required  SEP  with  linkage  of  SE  work 
products— such  as  the  requirements  in  the  RTM  and  TPMs— 
with  the  Integrated  Master  Schedule  and  work  packages. 

Acquisition  Reform 

Effective  acquisition  reform  is  a  stated  objective  of  DoD  and  of 
the  chairmen  of  the  Senate  and  House  Armed  Services  Com¬ 
mittees.  The  PMBOK  Guide  and  SEP  currently  are  "guidance." 
It  is  recommended  that  the  actual  reforms  impose  the  "guid¬ 
ance"  provided  above  as  contractual  requirements.  & 

The  author  can  be  contacted  at  paul.solomon@pb-ev.  com. 
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The  Loss  of  a  Leader  in  Defense  Acquisition 


CLAUDE  M.  BOLTON  JR. 


1945-2015 


jt 


The  Honorable  Claude  M.  Bolton  Jr.,  a  retired 
U.S.  Air  Force  (USAF)  major  general  and 
former  acquisition  official  for  the  USAF  and 
the  U.S.  Army  and  the  former  comman¬ 
dant  of  the  Defense  Systems  Management 
College  (DSMC),  died  unexpectedly  at  home  July 
28  at  age  69.  He  had  a  very  distinguished  40-year 
career  in  the  military  and  in  Department  of  Defense 
acquisitions. 

At  the  time  of  his  death,  Mr.  Bolton  was  Executive-in  Residence 
at  the  Defense  Acquisition  University,  helping  support  the  re¬ 
cruitment,  training  and  education  of  the  Defense  Acquisition 
Workforce.  Reflecting  his  lifelong  dedication  to  the  nation's 
defense,  he  once  told  an  interviewer:  "Our  soldiers  are  the  very 
best  in  the  world,  with  the  best  training  in  the  world.  I  figure 
the  least  we  can  do  is  make  sure  they  have  also  got  the  best 
equipment  in  the  world." 

He  served  as  mentor  to  a  great  many  officers  and  civilians 
and  often  would  officiate  at  promotion  and  retirement  cer¬ 
emonies.  At  these  events,  he  amazed  everyone  by  recollecting 
the  honoree's  family,  career  and  personal  interests— without 
the  aid  of  notes. 

A  veteran  of  more  than  32  years  of  active  military  service,  Mr. 
Bolton  retired  as  a  USAF  major  general  in  2002  and  was  As¬ 
sistant  Secretary  of  the  Army  for  Acquisition,  Logistics,  and 
Technology  from  2002  to  2008.  In  that  position,  Mr.  Bolton 
served  as  the  Army  Acquisition  Executive,  the  Senior  Procure¬ 
ment  Executive  and  the  Science  Advisor  to  the  Secretary  of 
the  Army.  He  also  was  the  senior  research  and  development 
official  for  the  Department  of  the  Army  and  had  principal 
responsibilities  for  all  logistics-related  matters.  He  provided 
oversight  for  the  life-cycle  management  and  sustainment  of 
Army  weapons  systems  and  equipment,  from  research  and 
development  through  test,  evaluation,  acquisition,  logistics, 
fielding  and  disposition.  In  addition,  Mr.  Bolton  oversaw  the 
Elimination  of  Chemical  Weapons  Program  and  had  oversight 
and  executive  authority  in  the  Project  and  Contracting  Office 
charged  with  Iraq's  reconstruction. 
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In  2000-2002,  he  was  commander  of  the  Air  Force  Secu¬ 
rity  Assistance  Center  and  (2000-2001)  assistant  to  the 
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commander  for  international  affairs  at  the  Air  Force  Mate¬ 
riel  Command  (AFMC)  at  Wright-Patterson  Air  Force  Base 
in  Ohio.  There  he  oversaw  more  than  $90  billion  in  military 
equipment  sales  to  foreign  countries.  In  June  1998  to  Octo¬ 
ber  2000,  he  was  program  executive  officer  for  fighter  and 
bomber  programs  in  the  Office  of  the  Assistant  Secretary 
of  the  Air  Force  (Acquisition).  In  June  1996  to  June  1998, 
he  was  director  of  requirements  at  the  AFMC,  where  also 
had  served  he  had  previously  served  in  September  1992  to 
March  1993  as  inspector  general.  Fie  was  DSMC's  com¬ 
mandant  from  March  1993  to  March  1996. 

Mr.  Bolton  was  born  in  December  1945  in  Sioux  City,  Iowa,  and 
received  his  bachelor's  degree  in  electrical  engineering  in  1969 
from  the  University  of  Nebraska-Lincoln,  where  he  also  served 
in  the  Reserve  Officers  Training  Corps.  Upon  graduating,  he 
entered  the  Air  Force  as  a  second  lieutenant,  where  he  served 
as  a  fighter  pilot  and  logged  more  than  2,700  hours  in  more 
than  30  different  aircraft  and  232  combat  missions— 40  of 
them  over  North  Vietnam. 

He  studied  acquisition  management  at  DSMC,  where  he  later 
was  commandant.  He  earned  a  master's  degree  in  manage¬ 
ment  from  Alabama's  Troy  State  University  in  1978,  and  a  mas¬ 
ter's  degree  in  national  security  and  strategic  studies  from  the 
Naval  War  College  in  Newport,  Rhode  Island,  in  1991.  He  also 
was  awarded  honorary  doctorates  from  the  United  Kingdom's 
Cranfield  University  in  2006,  and  from  his  undergraduate 
alma  mater,  the  University  of  Nebraska-Lincoln,  in  2007,  and 
completed  the  course  requirements  for  a  Ph.D.  in  electrical 
engineering  at  the  University  of  Florida. 

He  held  the  Defense  Distinguished  Service  Medal,  the  Legion 
of  Merit,  the  Distinguished  Flying  Cross  with  oak  leaf  cluster, 
the  Meritorious  Service  Medal  with  two  oak  leaf  clusters,  the 
Air  Medal  with  16  oak  leaf  clusters,  the  Vietnam  Service  Medal 
with  three  service  stars,  the  Republic  of  Vietnam  Gallantry 
Cross  and  the  Republic  of  Vietnam  Campaign  Medal. 

Mr.  Bolton  was  survived  by  his  wife  Linda  and  daughters 
Cindy  and  Jennifer,  five  grandchildren  and  five  siblings,  and 
his  parents. 

According  to  his  family,  Mr.  Bolton  was  intensely  dedicated  to 
the  tasks  that  he  took  on  and  was  known  for  "dreaming  big'' 
and  chasing  his  dreams  with  hard  work. 
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Delphi,  Dice 
and  Dominos 

Techniques 
to  Understand 
and  Mitigate 
Technical  Risk 


David  L.  Gallop 


t  is  no  secret  that  our  pro¬ 
grams  are  becoming  in¬ 
creasingly  complex  and 
interdependent.  With  that 
complexity  and  interdepen¬ 
dency  come  both  opportunity 
and  technical  risk.  There  are 
three  simple  techniques  any 
program  can  use  to  understand 
and  mitigate  technical  risk. 


•  • 


These  techniques  take  us  beyond  the 
common  (yet  important)  "risk  cube." 
They  can  be  used  together  or  sepa¬ 
rately.  They  are  designed  to  avoid 
three  common 
logic  traps:  fail¬ 
ing  to  account  for 
additional  per¬ 
spectives;  failing 
to  account  for  un¬ 
certainty  in  data; 
and  failing  to  ac¬ 
count  for  interde¬ 
pendencies.  The 
techniques  are 


9 


Jh  Gallop  is  a  professor  at 

_ . _ -  the  Defense  Systems 

Management  College , 
Fort  Belvoir,  Virginia.  He  is  a  Project  Management 
Professional  and  a  Certified  Systems  Engineering 
Professional. 


Defense  AT&L:  November-December  2015 


Consensus  is  not  our  goal. 

Our  goal  is  to  understand  the 
uncertainty.  This  requires  diverse 
perspectives. 


commonly  used  in  medicine,  finance  and  manufacturing. 
The  metaphors  we  use  are  Delphi,  dice  and  dominos. 

Delphi 

"Delphi"  refers  to  the  Oracle  of  Delphi,  where  in  ancient 
Greece  someone  (perhaps  ancient  program  managers)  went 
to  receive  prophecies  about  the  future.  Program  success  often 
depends  on  how  well  we  forecast  the  future.  We  often  rely  on 
teams  or  committees  to  provide  forecasts.  While  we  know 
groups  perform  better  than  their  best  member,  groups  have 
their  weaknesses.  Powerful  individuals  (personality  or  posi¬ 
tion)  can  dominate  and  therefore  limit  or  bias  a  forecast.  The 
Delphi  method  seeks  to  aggregate  independent  perspectives 
from  a  diverse  group.  You  can  find  Delphi  practiced  in  a  variety 
of  fields  such  as  environmental  management,  tourism,  educa¬ 
tion  and  marketing.  Program  managers  can  use  Delphi  in  situa¬ 
tions  of  complexity,  uncertainty  and  where  no  hard  facts  exist. 
Delphi  can  be  used  whenever  the  program  manager  needs 
to  analyze  disparate  perspectives  on  a  subject— contractor 
incentives,  task  durations,  nonmateriel  impacts,  etc.  In  the 
following  example,  we  will  use  Delphi  to  help  us  evaluate  a 
Technology  Readiness  Level  (TRL). 

TRLs  are  an  important  part  of  the  technical  risk  assessment 
process.  TRL  scores  measure  the  maturity  of  a  technology  on 
a  scale  of  1  through  9  (with  9  being  the  most  mature).  TRLs 


Figure  1.  Modified  Delphi  Process 


Steps 

Actions 

1 

Select  the  problem  or  topic  for  investigation. 

2 

Assemble  the  evidence/information  file  for  the 
Delphi  panel  members. 

3 

Select  the  Delphi  panel  members. 

4 

Send  the  Delphi  panel  members  evidence/infor¬ 
mation  file  to  be  reviewed  independently. 

5 

Receive  and  analyze  the  findings  from  the  Delphi 
panel. 

should  be  based  on  objective  evidence  instead  of  opinions. 
However,  even  the  best  available  evidence  can  be  incomplete 
and  subject  to  interpretation.  The  Delphi  technique  can  help 
us  increase  our  knowledge  and  provide  different  perspectives. 
The  classic  Delphi  technique  involves  using  multiple  rounds 
of  feedback  to  drive  the  group  toward  consensus.  This  can 
be  time  and  resource  intensive.  Classic  Delphi  also  has  been 
criticized  for  achieving  consensus  at  the  expense  of  the  best 
ideas.  We  will  use  Delphi's  power  to  extract  knowledge  from 
experts  while  avoiding  the  mentioned  disadvantages.  Figure 
1  summarizes  our  modified  Delphi  process. 

Once  the  problem  or  topic  area  is  identified,  assemble  the  evi¬ 
dence  file.  This  includes  the  objective  data  that  will  underpin 
the  TRL  score.  The  file  should  also  contain  information  regard¬ 
ing  the  operational  and  systems  context.  To  avoid  confirmation 
bias,  the  file  should  not  include  an  expected  TRL  score.  The  file 
should  have  plenty  of  "white  space"  to  elicit  information  and 
opinions  from  the  panel  members.  Possible  questions  include 
the  following:  What  additional  information  should  we  seek 
about  the  subject  technology?  Where  and  how  can  we  get 
that  information?  What  laws,  regulations  and  policies  must  we 
consider?  What  stakeholders  will  have  an  interest  in  this  tech¬ 
nology?  What  are  the  pitfalls  and  unintended  consequences 
of  this  technology?  What  are  some  materiel  and  nonmateriel 
alternatives  to  this  technology?  What  new  ideas  and  oppor¬ 
tunities  will  this  technology  offer?  Finally,  the  file  should  ask 
the  panel  members  to  provide  a  TRL  score  and  its  justification. 

Continuing  our  TRL  evaluation  example,  electing  the  Delphi 
panel  members  and  distributing  the  evidence  files  to  those 
members  are  the  next  steps.  Conventional  wisdom  would 
have  us  seek  out  the  most  brilliant  minds  in  the  domain  of 
the  subject  technology.  This  would  be  the  "perfectly,  perfect" 
panel  and  help  drive  panel  members  toward  a  consensus.  But 
consensus  is  not  our  goal. 

Our  goal  is  to  understand  the  uncertainty.  This  requires  diverse 
perspectives.  We  should  seek  the  "perfectly,  imperfect"  panel. 
Panel  members  should  have  collective  expertise  in  the  opera¬ 
tional  environment,  defense  acquisition  process,  technology 
development  and  the  specific  technical  domain. 


Figure  2.  TRL  Scores 
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Figure  3.  Monte  Carlo  Simulation 
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Delphi  panels  range  in  size  from  10  to  more  than  1,000. 
The  Delphi  process  does  not  call  for  the  panel  size  to  be 
representative  samples  for  statistical  purposes.  However, 
30  panel  responses  may  be  optimal  in  most  cases.  Each 
panel  member  responds  individually  and  independently.  If 
we  were  seeking  consensus,  we  would  perform  the  Delphi 
in  many  rounds.  However,  two  rounds  (initial  feedback  and 
follow-up  clarification)  should  suffice.  The  process  allows 
panel  members  the  freedom  to  think  reflectively  and  pro¬ 
pose  alternative  viewpoints. 

Finally,  collect  and  analyze  the  findings.  You  are  looking  for 
new  information,  risks,  stakeholders,  ideas  and  opportunities. 
The  increased  knowledge  and  perspective  from  the  Delphi 
method  can  lead  to  technical  excellence  and  innovation. 

Dice 

Program  managers  frequently  must 
assess  quantifiable  data— i.e.,  num¬ 
bers.  They  often  reach  conclusions  and 
make  decisions  without  considering  the 
uncertainty  in  the  numbers.  A  Monte 
Carlo  method  typically  runs  a  simulation 
many  times  to  obtain  the  distribution 
of  an  unknown  probabilistic  entity.  The 
Monte  Carlo  technique  can  be  used  in 
many  situations  such  as  cost  estimates, 
corrective  maintenance  task  durations 
or  risk  ratings.  Our  example  uses  the 
TRL  scores  from  our  Delphi  process  to 
assess  the  uncertainty. 

Figure  2  is  a  table  of  the  TRL  scores  from 
our  notional  Delphi  panel.  We  can  see 
that  the  scores  range  from  a  pessimistic 
TRL-4  to  an  optimistic  TRL-8.  The  most 
common  score  is  TRL-6.  To  arrive  at  a 
single  score,  we  could  simply  take  the 


average  of  the  scores.  The  problem  is  that  decisions  made 
based  on  the  average  are  wrong  on  average  ("The  Flaw  of 
Averages"  by  Sam  L.  Savage,  John  Wiley  &  Sons  publishers). 
Since  we  are  interested  in  understanding  the  uncertainty,  we 
should  consider  the  probability  distribution.  It's  time  to  roll  the 
dice  using  a  Monte  Carlo  simulation. 

There  are  many  powerful  commercial  software  packages  that 
will  perform  Monte  Carlo  simulations.  These  tools  can  be  ex¬ 
pensive  and  often  require  training.  Program  managers  need 
not  spend  a  great  deal  of  money  on  tools,  training  or  consul¬ 
tants.  Excel  is  widely  available  and  has  built-in  features  to  cre¬ 
ate  a  simulation.  The  Web  has  a  number  of  articles  and  videos 
that  provide  detailed  instructions  on  how  to  construct  a  Monte 
Carlo  simulation  in  Excel.  Thomas  and  Linda  McKee's  article, 
"Using  Excel  to  Perform  Monte  Carlo  Simulations,"  in  the  De¬ 
cember  2014  issue  of  Strategic  Finance ,  is  a  great  resource. 

We  created  a  simulation  in  Excel  that  ran  100  iterations  (Figure 
3).  In  that  simulation,  we  assigned  random  probabilities  that 
the  actual  TRL  lies  somewhere  between  the  minimum  (TRL-4) 
and  maximum  (TRL-8),  with  a  greater  probability  around  the 
mode— or  most  frequent— (TRL-6).  Many  random  processes 
follow  a  normal  (bell-shaped)  distribution,  but  some  do  not. 
Since  we  have  a  minimum,  maximum  and  a  mode,  we  used  a 
triangular  distribution.  The  McKee  article  mentioned  earlier 
describes  the  steps  to  create  several  common  probability  dis¬ 
tributions  in  Excel. 

This  simulation  reveals  a  strong  probability  that  the  technol¬ 
ogy  maturity  actually  is  TRL-5.  If  we  assume  that  the  program 
manager  expected  the  maturity  to  be  TRL-6,  there  should  be 
mitigation  commensurate  with  the  risk  that  the  maturity  ac¬ 
tually  is  TRL-5.  The  mitigation  could  include  gathering  more 
information  or  exploring  alternative  solutions.  So  far,  we  have 
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Our  goal  is  to  keep  all  the  dominos 
standing ...  but  we  never  have  the 
resources  to  monitor  and  control 
everything  equally. 


used  the  Delphi  method  to  consider  perspectives  and  the  dice 
(Monte  Carlo  simulation)  to  consider  uncertainty.  Our  next 
tool  will  allow  us  to  consider  interdependencies. 

Dominos 

Consider  all  the  risks  in  a  program.  We  can  group  those  risks 
into  four  categories— cost,  schedule,  technical  and  program¬ 
matic.  Figure  4,  adapted  from  page  222  of  the  International 
Council  on  Systems  Engineering's  2011  Systems  Engineering 
Handbook  (INCOSE-TP-2003-002-03.2.2),  shows  the  typi¬ 
cal  relationships  among  the  risk  categories.  Visualize  these 
risks  as  groups  of  dominos  standing  on  end.  One  domino 
falling  may  trigger  a  cascade  of  issues  across  the  program. 
Our  goal  is  to  keep  all  the  dominos  standing  ...  but  we  never 
have  the  resources  to  monitor  and  control  everything  equally. 
It  also  is  difficult  to  predict  what  other  dominos  will  fall  if 
a  domino  tips  over.  If  only  we  had  a  tool  that  would  help 
program  managers  focus  their  resources  and  predict  where 
downstream  issues  could  occur  from  a  falling  domino.  An 
N-squared  chart  can  help. 

An  N-squared  chart  can  map  interdependencies  of  functions, 
components,  documents,  organizations  and  budget  lines  ... 
just  about  anything  that  can  be  decomposed  into  smaller  units 
and  where  those  smaller  units  have  exchanges.  To  build  an  N- 
squared  chart,  put  the  elements  (to  be  designated 
by  the  letter  "E"  and  given  identifying  numbers) 
in  the  diagonal  blocks  of  a  matrix.  The  exchanges 
between  the  two  elements  appear  in  the  intersec¬ 
tion  of  the  corresponding  row  and  column.  In  the 
generalized  example  in  Figure  5,  El  receives  an 
external  input  and  an  input  from  E2.  El  provides 
an  external  output  and  an  output  to  E2. 

Let's  return  to  our  domino  analogy.  If  the  E2  dom¬ 
ino  falls,  the  El  domino  could  fall  as  well.  The  N- 


squared  chart's  predictive  power  increases  as  we  provide  more 
detail  on  the  interdependencies.  From  Figure  6,  we  can  infer 
that  El  and  E4  deserve  our  attention.  El  is  the  interface  with  an 
external  system,  outputs  data  for  three  elements,  and  receives 
inputs  from  five  elements.  E4  receives  four  inputs  from  three 
elements.  We  also  can  see  that  three  elements  depend  on 
Data  Item  A.  Data  Item  A  also  deserves  our  attention.  The  fact 
that  El  outputs  Data  Item  A  underscores  the  importance  of 
the  El  domino  remaining  upright.  As  a  predictive  tool,  a  perfor¬ 
mance  shortfall  in  E6  would  predict  a  performance  shortfall  in 
El  (and  in  turn  shortfalls  in  E2, 3, 4,  and  the  external  system). 
All  the  dominos  are  important ...  but  some  dominos  are  more 
important  than  others. 

Conclusion 

Understanding  and  mitigating  technical  risk  requires  cast¬ 
ing  a  wide  net  for  information  and  perspectives.  The  Delphi 
technique  is  useful  in  complex  situations  where  there  may 
be  no  clear  choice.  Monte  Carlo  simulation  highlights  the 
uncertainty  in  data.  Knowing  this  uncertainty  allows  com¬ 
munication  of  the  confidence  in  the  data.  The  N-squared 
chart  maps  the  interdependencies  so  we  focus  our  monitor¬ 
ing  and  control.  & 

The  author  can  be  contacted  at  david.gallop@dau.mil. 


Figure  6.  N-Squared  Chart 
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Figure  5.  The  Elements  Interact 
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If  Your  Technology  Works,  Will  It  Matter? 


Ryan  Umstattd,  Ph.D. 


The  foundational  question  above  is  asked  and  then  an¬ 
swered  by  our  program  directors  before  they  launch  any 
new  focused  initiative  at  the  U.S.  Department  of  Energy's 
Advanced  Research  Projects  Agency  (ARPA-E). 

It's  simply  not  good  enough  to  perform  groundbreaking  applied  research  if  we 
haven't  also  developed  a  credible  path  toward  the  commercial  market  for  our  technologies. 
While  the  U.S.  Department  of  Defense  (DoD)  Research,  Development,  Test  and  Evaluation 
construct  is  not  geared  toward  placing  products  in  the  broad  commercial  marketplace,  many 
lessons  learned  from  asking  that  foundational  question  remain  valuable  in  managingthe  DoD's 
science  and  technology  investments. 
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Managing  More  Than  Technology  Development 

Our  mission  at  ARPA-E  is  to  catalyze  and  support  the  development  of  transformational,  high- 
impact  energy  technologies.  That  said,  a  technological  breakthrough  by  itself  may  not  be  suf¬ 
ficient  to  drive  a  transformation  that  becomes  truly  disruptive  (see  Figure  1).  Bringing  a  new 
technology  to  the  tipping  point  of  disruption  involves  success  on  several  fronts,  including  science 
and  technology  development,  a  viable  business  model  and  value  proposition,  favorable  market 


Umstattd  is  a  senior  commercialization  advisor  at  the  Department  of  Energy's  Advanced  Research  Projects  Ag 
Energy  ( ARPA-E )  where  he  helps  prepare  breakthrough  energy  technologies  for  transition  from  the  lab  to  customers 
within  the  Department  of  Defense.  Earlier ;  he  spent  20  years  developing  and  delivering  technologies  as  a  U.S.  Air  Force 
acguisitions  officer.  He  has  a  Ph.D.  in  applied  science  (plasma  physics). 


Figure  1.  Technology  Transformation  and  Disruption 
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conditions,  and  financial  support  for  scale-up  and  production. 
As  a  government  agency  that  provides  funding,  we  have  the 
role  of  developing  technologies  until  they  are  ready  for  market 
decisions;  to  prepare  for  that  stage,  we  manage  and  guide  our 
awardees  in  both  technical  and  commercial  development.  We 
employ  both  program  directors  who  manage  the  technology 
development  of  various  projects  as  well  as  project-specific, 
dedicated  technology-to-market  advisors.  Our  tech-to-market 
advisors  help  our  awardees  plan  and  execute  the  many  aspects 
of  commercialization  that  go  beyond  successful  technology 
development.  As  shown  in  Figure  2,  the  tech-to-market  team 
provides  a  foundation  that  supports  the  program  director's 
advanced  technology  effort  as  part  of  our  strategy  to  increase 
the  yield  of  good  ideas  that  become  impactful  products.  Our 
program  directors  and  tech-to-market  advisors  work  together 
to  build  a  community  of  advocates  ranging  from  the  labora¬ 
tory  to  the  marketplace  so  that  successfully  demonstrated, 
transformative  technologies  have  a  better  chance  of  being 
embraced  by  potential  users. 

There  can  be  a  natural,  healthy  tug-of-war  between  our  pro¬ 
gram  directors  and  our  tech-to-market  team  over  technical 
risk.  While  the  program  directors  are  asked  to  identify  and 
fund  high-risk,  potentially  game-changing  research,  the  tech- 
to-market  advisors  will  need  to  demonstrate  that  the  asso¬ 
ciated  technical  risk  has  been  driven  low  enough  to  capture 
the  interest  of  potential  commercialization  partners.  At  the 
end  of  the  day,  because  our  program  directors  and  tech-to- 
market  advisors  are  both  in  hot  pursuit  of  a  positive  response 
to  our  mantra,  "If  it  works,  will  it  matter?,"  they  appreciate  each 
other's  perspective  and  remain  unified  by  that  common  goal. 

Supplementing  Awardees’  Skill  Mix 

Because  our  goal  is  to  enable  the  project  teams  to  man¬ 
age  their  own  paths  to  commercialization,  our  tech-to- 


market  efforts  must 
extend  beyond  the 
ARPA-E  program  and 
tech-to-market  team. 
As  part  of  an  ARPA-E 
award,  each  awardee  is 
asked  to  assign  a  tech- 
to-market  lead  person 
who  establishes  the 
project's  commercial¬ 
ization  goals  that  drive 
the  technology  along 
the  path  to  market.  In 
fact,  a  portion  of  the 
award  funding  must  be 
designated  for  perform¬ 
ing  technology  transfer 
and  outreach.  A  skilled 
tech-to-market  lead  on 
the  awardee  team  can 
make  the  difference 
between  a  transforma¬ 
tional  idea  that  withers  on  the  vine  and  a  disruptive  new 
technology  that  overtakes  the  competition  and  dominates 
the  market  (as  per  Figure  1).  An  effective  tech-to-market 
lead  must  be  intimately  familiar  with  the  technology  land¬ 
scape  and  the  technical  details  of  the  project,  so  sometimes 
the  tech-to-market  lead  also  is  the  project's  lead  scientist 
or  principal  investigator.  The  skill  set  needed  for  performing 
bread-and-butter  commercialization  tasks  is  quite  different, 
however,  from  that  needed  for  performing  world-class  ap¬ 
plied  research.  Thus,  a  principal  investigator  is  encouraged 
to  seek  out  a  tech-to-market  lead  with  a  strong  background 
in  not  just  the  technical  area  of  the  project  but  also  in  pre¬ 
paring  business  cases,  value  propositions  and  intellectual 
property  protection  strategies. 

At  each  quarterly  project  review,  the  ARPA-E  program  direc¬ 
tor  and  tech-to-market  lead  actively  engage  with  the  awardee 
team  to  understand,  direct  and  advise  both  technical  and  tech- 
to-market  efforts.  This  government-awardee-combined  team 
becomes  a  powerful  force  when  each  group  is  committed  to 
making  the  technology  matter. 

Transitioning  Technology 

Each  awardee's  progress  is  monitored  against  carefully  de¬ 
signed  technical  and  tech  transfer  milestones— but  meeting 
those  milestones  alone  does  not  necessarily  mean  theirtech- 
nology  will  be  commercialized.  The  government  and  awardee 
teams  should  work  together  throughout  the  project  term  to 
identify  and  connect  with  potential  transition  partners.  As 
each  of  our  projects  come  to  a  close,  we  aim  to  keep  the  suc¬ 
cessful  technologies  moving  toward  commercialization  via 
several  available  paths,  including: 

■  New  company  formation 

■  Strategic  partnership  with  an  existing  company 
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Figure  2.  Bringing  Technology  to  Market 
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When  the  awardee's  team  has  a  strong  technical  success, 
the  right  personnel  and  supportive  market  conditions,  a  new 
business  may  be  spun  off  at  the  conclusion  of  the  project  as 
the  path  to  commercialization.  Another  route  leverages  the 
experience  and  size  of  an  existing  company  by  attracting  its 
interest  in  the  new  technology.  If  the  technical  risk  remains 
too  high  for  either  of  these  two  paths,  the  project  team  might 
seek  out  follow-on  funding  from  private  investors  or  another 
government  agency  in  order  to  further  mature  the  technology. 
Regular  and  open  communication  between  the  government 
and  awardee  teams  is  crucial  to  understanding  the  successes 
and  stumbling  blocks  along  a  transition  path.  To  date,  proj¬ 
ects  funded  by  ARPA-E  have  resulted  in  the  formation  of  at 
least  30  new  companies,  attracted  more  than  $850  million 
of  private  sector  follow-on  funding  and  led  to  more  than  37 
projects  funded  via  partnership  with  other  parts  of  federal, 
state  or  local  government. 

It  also  is  important  during  transition  to  find  the  right  partners 
who  can  add  skills  and  resources  beyond  those  of  the  original 
project  team.  Many  ARPA-E  projects  focus  on  breakthrough, 
building-block  technologies  that  will  need  to  be  incorporated 
into  a  larger  system  in  order  to  fully  commercialize,  such  as 
actuators  for  sun-tracking  solar  panels,  cell-level  battery  di¬ 
agnostics  to  improve  energy  management  or  higher  current 
density  superconducting  wire.  Thus,  once  a  project  team  has 
identified  that  crucial  first  market,  an  important  next  step  may 
be  to  find  a  transition  partner  who  can  help  the  team  plan  and 
execute  the  effort  that  takes  the  project  beyond  a  lab  proto¬ 
type  of  a  component  to  a  full-blown  system  demonstration. 


For  the  subset  of  ARPA- 
E  projects  that  also  have 
clear  defense-specific 
applications,  we  encour¬ 
age  the  awardees  to  forge 
partnerships  with  a  de¬ 
fense  customer— ideally 
by  affordably  addressing  a 
formal  requirement.  It  also 
is  beneficial  for  the  project 
team  to  consider  a  part¬ 
nership  with  a  commer¬ 
cial  firm  that  has  a  proven 
track  record  of  being  able 
to  contract  with,  manu¬ 
facture  for  and  deliver 
systems  to  the  DoD. 

Finally,  as  you  have  likely 
seen  in  your  acquisition 
careers,  the  technologies 
with  the  greatest  likeli¬ 
hood  of  making  it  through 
the  defense  acquisition 
system  are  supported  by  a  combination  of  both  technology 
push  and  requirements  pull.  While  ARPA-E  does  not  have 
formally  documented  user  requirements,  our  tech-to-market 
program  includes  activities  designed  to  help  awardees  under¬ 
stand  the  current  market  needs,  generate  customer  pull  and 
increase  their  odds  at  having  their  new  technologies  matter. 

Parting  Thoughts 

While  we  apply  the  tips  above  in  managing,  executing  and 
partnering  across  our  entire  ARPA-E  project  portfolio,  note 
that  these  insights  are  much  more  readily  applied  in  defense 
science  and  technology  development  for  applied  research  and 
beyond.  The  fundamental  science  research  performed  can¬ 
not  benefit  nearly  as  much,  given  that  a  market  cannot  be 
defined  when  the  end-goal  technology  is  still  in  flux.  For  each 
office  that  funds,  manages  or  performs  applied  research,  we 
encourage  you  to  find  your  most  natural  and  effective  tran¬ 
sition  partners  and  keep  them  informed  and  engaged  often! 

For  ARPA-E,  success  lies  in  having  effective  partnerships  not 
just  internally  between  our  program  directors  and  tech-to- 
market  advisors,  but  also  externally  with  our  counterparts 
on  the  awardee  teams  and  potential  transition  partners.  Of 
course,  if  achieving  successful  technology  transition  were  as 
simple  as  following  the  guidelines  above,  there  would  be  a 
straightforward  checklist  resulting  in  higher  yields  from  lab 
to  market.  From  what  we  have  seen  here  at  ARPA-E,  these 
guidelines  are  executed  by  top-notch  personnel  attracted  by 
an  organizational  environment  that  is  transparent  and  eager  to 
experiment  with  new  approaches  to  developing  and  delivering 
energy  technology  that  matters. 

The  author  can  be  reached  at  ryan.umstattd@hq.doe.gov. 
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One  of  the  most  difficult  transitions  in  a  product's  life  cycle  is  the  transition  from  the 
science  and  technology  (S&T)  environment  to  advanced  development  (AD).  Transition 
planning  is  necessary  to  bridge  this  technology  "Valley  of  Death"  in  which  promising 
technologies  frequently  are  delayed  or  fail  to  make  the  transition.  Without  successful 
transitions,  intellectual  and  financial  investments  in  research  do  not  translate  to  im¬ 
proved  capabilities  for  the  U.S.  military.  Early  and  thorough  transition  planning  is  key  to  success. 

Transitions  in  medical  product  development  traditionally  have  been  conducted  within  the  military  Services,  moving 
from  a  Service  S&T  program  to  its  respective  AD  program.  The  launch  of  the  Defense  Health  Program  (DHP)  in 
2008  provided  jointly  managed  resources  to  supplement  and  leverage  existing  individual  Service  research-and- 
development  (R&D)  investments  and  also  provided  the  impetus  to  establish  joint  medical  development  portfolios. 

In  2010,  the  Joint  Program  Committee  for  Combat  Casualty  Care  established  the  Joint  Hemorrhage  and  Resus¬ 
citation  R&D  Portfolio,  which  provided  the  first-ever  comprehensive  Department  of  Defense  (DoD)  view  of  the 
R&D  pipeline  for  medical  products  that  address  bleeding,  the  leading  cause  of  potentially  preventable  deaths  on 
the  battlefield  (Eastridge  et  al.,  2012).  Initial  pipeline  review  revealed  several  promising  technologies  in  late  S&T 


Pusateri  is  Portfolio  Manager  for  the  Department  of  Defense  Hemorrhage  and  Resuscitation  Re¬ 
search  and  Development  Program  for  the  Combat  Casualty  Care  Research  Program  at  the  U.S.  Army 
Medical  Research  and  Materiel  Command  in  Fort  Detrick ,  Maryland.  Macdonald  is  Product 
Manager  for  Pharmaceutical  Systems  at  the  U.S.  Army  Medical  Materiel  Development 
Activity ,  Fort  Detrick.  He  is  a  blood  products  expert.  Given  is  a  program  officer  in 
the  Office  of  Naval  Research  in  Arlington,  Virginia.  Walter,  a  retired  U.S. 

Air  Force  lieutenant  colonel,  is  the  Air  Force  Medical  Support  Agency 
Advanced  Development  Liaison  Field  Engineer  in  Falls  Church, 

Virginia.  Prusaczyk  is  Director  of  Acquisition  and  Pro¬ 
gram  Management  at  the  Naval  Medical  Re¬ 
search  Center  in  Silver  Spring,  Maryland. 

!' 

I 

,  *  1  *,  I*  '  ;  {  1 


41 


Defense  AT&L:  November-December  2015 


that  did  not  have  established  transition  pathways,  Program 
Executive  Office  buy-in,  or  adequate  funding.  While  there 
were  significant  opportunities  to  leverage  funding  and  other 
resources  across  the  DoD,  no  process  had  been  established 
for  joint  transitions  to  medical  AD.  Therefore,  our  team,  with 
input  from  a  number  of  stakeholders  and  subject-matter  ex¬ 
perts  (SMEs),  developed  a  Joint  Transition  Planning  Process 
that  successfully  facilitated  transitions  for  seven  programs. 

Product  Development  “Valley  of  Death” 

The  transition  from  S&T  to  medical  AD  within  DoD  roughly 
parallels  what  has  been  termed  the  pharmaceutical  or  technol¬ 
ogy  "Valley  of  Death."  The  "Valley  of  Death"  most  frequently 


FDA  requirements.  Some  programs  continue  through  Phase 
I  clinical  trials,  while  others  may  end  before  completion  of 
preclinical  development. 

■  Different  Services  have  different  procedures  and  expecta¬ 
tions  for  transition. 

■  Stand-alone  Service  programs  may  not  recognize  opportuni¬ 
ties  to  leverage  funding  or  to  fill  joint  (common)  capability 
gaps. 

■  S&T  teams  often  are  not  aware  of  the  types  of  information 
and  data  needed  for  advanced  developers  to  accept  a  pro¬ 
gram,  or  to  enable  entry  into  FDA  regulated  trials. 

■  Awareness,  communication  and  coordination  may  be  mini¬ 
mal  among  Service  S&T  and  AD  programs. 


It  is  important 

early  in  the  process  to  reconcile 


development  processes  as  much  as  possible 
across  Services  and  to  clarify  lines  of  authority 
within  different  Service  paradigms. 


refers  to  a  lack  of  funding  or  development  partners  for  a  prod¬ 
uct  to  bridge  the  technology  transition  phase  (Figure  1).  This 
critical  transition  period  spans  from  the  late  research,  pre¬ 
materiel  development  decision  (MDD)  phase,  through  tech¬ 
nology  development.  Specific  activities  during  this  phase  for 
drugs  or  biologies  products,  for  example,  include:  final  proof 
of  concept  in  relevant  animal  models;  manufacturing  develop¬ 
ment  in  accordance  with  Food  and  Drug  Administration  (FDA) 
Good  Manufacturing  Practices;  analytical  development;  ani¬ 
mal  safety  studies;  initial  clinical  development  planning;  Inves¬ 
tigational  New  Drug  (IND)  Application  filing,  first  in  human 
safety  studies;  and  others.  The  proportion  of  these  activities 
before  transition  to  a  chartered  AD  Integrated  Product  Team 
(IPT)  depends  on  the  Service  sponsoring  the  S&J  and  the  spe¬ 
cifics  of  the  program. 


Joint  Transition  Planning  Process 

A  Joint  Transition  Planning  Process  was  developed  to  facilitate 
transitions  involving  different  funding  sources,  different  Ser¬ 
vice  and  agency  paradigms  for  S&T,  and  different  experiences 
with  AD.  This  process  helps  bridge  the  technology  develop¬ 
ment  "Valley  of  Death"  by  facilitating  and  tailoring  late-stage 
S&T  to  position  the  product  for  transition  to  AD  and  generating 
information  useful  in  higher-level  acquisition  decisions.  It  com¬ 
plements  but  does  not  replace  Service  planning  processes.  It 
is  important  early  in  the  process  to  reconcile  development 
processes  as  much  as  possible  across  Services  and  to  clarify 
lines  of  authority  within  different  Service  paradigms.  Key  com¬ 
ponents  are  the  Joint  Transition  Planning  Meeting  and  the  Joint 
Transition  Working  Group  (JTWG),  and  there  is  an  overarching 
theme  of  communication. 


The  most  significant  opportunity  in  developing  a  joint  transi¬ 
tion  process  for  medical  development  programs  is  for  Service 
S&T  programs  to  directly  transition  into  joint  AD  programs, 
with  supplemental  funding  provided  via  the  DHP.  Joint  pro¬ 
grams  provide  an  opportunity  to  systematically  address  ca¬ 
pability  gaps  that  are  shared  by  all  Services  and  to  leverage 
Service  funding  and  capabilities.  A  number  of  challenges  also 
are  apparent: 

■  S&T  programs  from  various  Services  and  agencies  typically 
end  at  slightly  different  phases  of  research  with  respect  to 


Joint  Transition  Planning  Meeting 

The  Joint  Transition  Planning  Meeting  provides  a  forum  for 
S&T  team  presentation  and  a  process  for  building  advanced 
developer  awareness.  The  meeting  is  structured  to  provide 
the  following  information  and  functions. 

■  Provide  current  product/program  information. 

■  Provide  information  and  assistance  to  the  S&T  team  to 
prepare  for  transition. 

■  Assess  potential  for  transition. 

■  Identify  potential  Service  interest  as  AD  lead. 
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Figure  1.  DoD  Medical  Product  Development  Milestones 
and  Pharmaceutical  Development  program 
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Source:  U.S.  Army  Medical  Research  and  Materiel  Command ,  Fort  Detrick ,  Maryland ,  Decision  Gate  Office  (Modified). 


•  Identify  funding  source(s). 

■  Understand  Service(s)  concurrence  with  importance  of 
capability  and  approach. 

Meeting  preparation  includes  assisting  and  advising  the  S&T 
team  and  providing  specific  presentation  content  guidelines. 
The  primary  meeting  outcomes  are  recommendations  and 
may  include  identifying  a  lead  advanced  developer  and  funding 
commitment  (subject  to  senior  leader  and  milestone  decision 
authority  [MDA]  approval)  or  formation  of  a  JTWG  to  facili¬ 
tate  movement  toward  AD  and  a  future  reassessment.  It  also 
may  be  recommended  that  the  S&T  program  continue  in  S&T 
or  that  that  it  be  discontinued  due  to  lack  of  Service  interest 
or  funding. 

The  JTWG 

A  JTWG  is  formed  to  facilitate  the  transition  of  a  promising 
S&T  program  to  a  joint  AD  program.  The  working  group  per¬ 
forms  functions  similar  to  those  of  an  IPT  for  a  limited  pe¬ 
riod  to  bridge  the  gap  until  official  designation  of  an  AD  lead, 
a  chartered  IPT  and  a  formal  AD  program.  The  roles  of  the 
JTWG  can  be  tailored  to  the  needs  of  the  specific  program 
and  transition  situation. 

Key  functions  and  activities  include  the  following  as  needed: 

■  Perform  a  detailed  assessment  of  current  program  status 
(e.g.,  cost,  schedule,  regulatory,  technical  feasibility,  risk 
analysis). 


■  Perform  analysis  of  alternatives. 

■  Review  the  currently  planned  regulatory  pathway  and 
recommend  potential  changes. 

■  Determine  status  of  requirements  and  acquisition  docu¬ 
ments  and  initiate  production  of  documents. 

■  Assess  current  contracts  and  develop  estimates  for  modi¬ 
fications  and/or  future  contracts. 

■  Develop  budget  estimates  and  determine  availability  of 
funding. 

■  Develop  an  updated  estimated  timeline  and  work  break¬ 
down  structure. 

■  Provide  recommendations  regarding  a  development 
decision. 

■  Assess  relevant  capability  gaps,  potential  methods  of 
employment  and  implications  for  an  acquisition  strategy. 

■  Assess  relevance  of  the  product  for  each  Service. 

■  Assess  security  and/or  status  of  intellectual  property. 

■  Discuss  the  role  of  industry  partners. 

Early  activities  include  an  initial  forming  meeting,  initiation 
of  periodic  meetings,  an  in-progress  review  (IPR)  within 
60  to  90  days,  and  establishment  of  electronic  information 
sharing.  Activities  continue  under  the  JTWG  until  either 
an  advanced  developer  is  designated  and  a  chartered  IPT 
formed  or  a  decision  is  made  not  to  continue  the  product 
into  AD.  Activities  are  tailored  to  provide  the  information 
needed  to  enable  initial  assessments,  recommendations 
and  decisions.  The  focus  is  on  preparing  for  transition  and 
enabling  post-transition  activities. 
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JTWG  Membership 

The  JTWG  is  not  an  officially  char¬ 
tered  body.  Rather,  it  is  a  working 
group  that  technically  functions  as  a 
subgroup  of  the  joint  portfolio-level 
steering  committee.  Core  member¬ 
ship  is  augmented  over  time  by  ad¬ 
ditional  experts  and  representatives. 

Membership  includes  representation 
from  S&T  team,  Services  or  agencies, 
advanced  developers,  regulatory  sci¬ 
entists  and  others  as  needed  (e.g., 
requirements,  budget,  legal,  contract¬ 
ing,  cost  estimating,  scientific  or  user 
medical  SMEs,  logistics  personnel). 

Experience  to  Date 

The  process  is  flexible  and  structured 
sufficiently  to  ensure  that  the  proper 
information  is  available  to  the  appro¬ 
priate  groups  to  facilitate  planning 
and  decision  making.  Initially,  there 
was  significant  controversy  over  what 
milestone  decision  process  and  what 
contracting  and  regulatory  oversight 
office(s)  to  use,  who  would  serve  as 
MDA,  and  other  variables.  While  these  were  legitimate  and 
important  issues,  they  were  largely  due  to  the  fact  that  the 
Defense  Health  Agency  (DHA)  was  being  established  and  the 
potential  impact  on  procedural  requirements  was  not  yet  clear. 
Each  Service  has  established  processes  for  within-Service 
transitions.  The  goal  was  not  to  establish  new  requirements 
to  which  programs  must  conform  but  to  develop  a  process 
to  move  programs  forward  in  a  way  that  makes  transitions 
compatible  with  entry  into  the  AD  processes  already  in  place 
for  each  Service.  The  process  has  facilitated  the  progress  of 
several  products  and  also  a  decision  to  discontinue  a  program. 
Our  experience  to  date  includes: 

2011—  The  Red  Cell  Pharming  Program:  This  S&T  program 
was  sponsored  by  the  Defense  Advanced  Research  Projects 
Agency  (DARPA)  to  develop  a  technology  to  produce  univer¬ 
sal  red  blood  cells  in  vitro  and  to  eliminate  the  need  to  col¬ 
lect  red  cells  from  donors.  This  would  reduce  logistical  con¬ 
straints  and  enhance  the  safety  of  blood  transfusions.  At  the 
Joint  Transition  Planning  Meeting,  it  was  concluded  that  the 
projected  unit  cost  for  the  "Pharmed"  red  cells  was  prohibitive. 
The  program  was  not  moved  into  AD.  This  promising  technol¬ 
ogy  now  is  being  explored  for  other  applications. 

2012—  Solvent  Detergent  Spray-Dried  Plasma:  This  S&T 
program  was  sponsored  by  the  Office  of  Naval  Research 
(ONR)  and  Marine  Corps  Systems  Command  to  produce  a 
dried  plasma  that  would  reduce  dramatically  the  logistical 
constraints  associated  with  current  frozen  plasma.  A  dried 
product  that  could  be  rehydrated  when  needed  would  make  it 
possible  to  provide  plasma  transfusions  for  combat  casualties 


wherever  medically  needed  on  the  battlefield  or  in  transport, 
as  opposed  to  only  where  freezers  and  thawing  equipment 
are  present.  This  program  was  transitioned  as  the  first  Joint 
Medical  Advanced  Development  Program  chartered  by  DHA. 
The  program  was  guided  through  the  transition  by  a  JTWG  for 
more  than  a  year,  until  an  IPT  was  chartered.  It  is  now  a  Navy- 
led,  Joint  Advanced  Development  Program,  funded  by  DHP 
and  Navy,  with  significant  program  support  from  the  Air  Force. 
This  is  part  of  a  three-product  U.S.  Government  strategy  to 
develop  dried  plasma,  including  programs  that  incorporate 
different  technological  approaches  and  that  are  sponsored 
by  the  Army  and  the  Biomedical  Advanced  Research  and  De¬ 
velopment  Authority  (BARDA).  Coordination  is  facilitated  by 
IPT  cross-membership. 

2013— The  Wound  Stasis  System:  This  DARPA  program  was 
designed  to  develop  an  expanding  foam  that  could  be  infused 
into  the  abdominal  cavity  to  control  internal  bleeding  until  the 
injured  Service  member  could  reach  a  surgeon.  The  program 
successfully  advanced  under  the  guidance  of  a  JTWG  and  is 
now  an  Army-led,  Joint  Advanced  Development  Program, 
funded  by  DHP  and  the  Army. 

2013— Platelet  Derived  Hemostatic  Agent:  There  were  two 
competing  technologies  and  development  programs— one 
S&T  program  sponsored  by  Army  and  DARPA,  the  other  by 
ONR.  The  goal  was  a  product  that  could  be  infused  intrave¬ 
nously  to  help  stop  bleeding.  The  recommendation  from  the 
Joint  Transition  Planning  Meeting  was  to  move  both  products 
forward  for  a  later  down-select,  under  an  Army-led  Joint  Ad¬ 
vanced  Development  Program.  Subsequently,  our  interagency 
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DoD  X-Stat  Dressing  to  stop  bleeding  in  deep  wound  tracks. 

Photo  (with  permission)  by  RevMedx  Inc.,  Wilsonville,  Oregon. 


partner,  BARDA,  agreed  to  develop  one  product,  while  the 
Army  continued  development  of  the  other.  The  two  programs 
are  progressing  with  close-coordination  and  cross-member¬ 
ship  on  the  IPTs.  The  DoD  program  is  funded  by  DHP. 

2013— Valproic  Acid:  This  S&T  program  was  sponsored  by 
ONR  to  develop  a  drug  that  could  be  injected  into  a  combat 
casualty  to  stabilize  affected  tissues  and  increase  survival  time 
before  reaching  surgery  and  blood  transfusion.  Following  the 
Joint  Transition  Planning  Meeting,  a  JTWG  was  formed  and 
continues  to  move  the  program  forward  until  a  Navy-led  Joint 
Advanced  Development  Program  is  established.  The  program 
continues  with  DHP  and  Navy  funding  and  significant  program 
support  from  the  Air  Force. 

2013— Surviving  Blood  Loss  Program:  The  goal  of  this 
DARPA  S&T  program  was  to  develop  a  low-volume  treatment 
that  could  be  administered  to  bleeding  casualties  to  increase 
survival  time  after  severe  blood  loss.  The  research  program 
developed  a  new  drug— ethinyl  estradiol-3-sulfate.  At  the  Joint 
Transition  Planning  Meeting,  it  was  determined  that  additional 
work  was  needed  before  any  Service  would  be  willing  to  com¬ 
mit  to  leading  a  Joint  Advanced  Development  Team.  Currently, 
the  program  is  proceeding  under  the  guidance  of  a  JTWG  in 
close  coordination  with  DARPA  and  the  participating  DoD  lab. 
Funding  is  provided  by  DHP  and  DARPA,  and  the  program  will 
be  reassessed  in  a  year. 


2013  —X-Stat  Dressing:  This  product  is  the  result  of  a  S&T 
effort  led  by  the  U.S.  Special  Operations  Command  (US- 
SOCOM)  to  develop  a  hemostatic  dressing  to  stop  bleed¬ 
ing  from  deep  wound  tracts  in  areas  difficult  to  reach  with 
standard  dressings  or  tourniquets.  A  JTWG  was  formed  and 
rapidly  transitioned  to  a  chartered  IPT  and  an  Army-led  Joint 
Advanced  Development  Program.  In  addition  to  USSOCOM 
funding,  the  program  has  received  important  support  from 
both  the  Air  Force  and  the  Army.  The  first-generation  product 
has  been  FDA  approved  for  battlefield  use  and  has  undergone 
limited  fielding.  The  ongoing  development  program  is  aimed  at 
gaining  broader  FDA  approval  and  information  on  clinical  use. 

Status  of  the  Process 

Our  process  has  met  with  a  number  of  challenges  and  has 
been  refined  over  time.  Some  aspects  of  developing  the 
process  and  building  support  have  been  more  difficult  than 
anticipated.  At  times,  some  misunderstandings  have  caused 
controversy  and  resistance.  During  the  first  year  or  two  when 
we  were  developing  and  implementing  this  process,  there  were 
concerns  that  we  were  trying  to  bypass  established  acquisi¬ 
tion  and  milestone  processes  and  to  move  programs  forward 
without  proper  programmatic  and  contracting  oversight.  We 
addressed  these  concerns  by  increasing  communication,  by 
documenting  our  oversight  processes,  and  by  seeking  addi¬ 
tional  guidance  where  needed. 

Bringing  the  Services  together  to  cooperate  on  programs  has 
been  easier  than  initially  expected.  The  initial  incentive  for  par¬ 
ticipation  was  the  potential  to  leverage  DHP  funding.  However, 
as  the  process  evolved  and  achieved  some  early  successes,  a 
belief  in  the  importance  of  the  process  seemed  to  dominate. 
In  fact,  after  a  recent  budget  cut,  two  projects  were  almost 
completely  defunded  in  future  years.  Nonetheless,  the  JTWGs 
continued  to  meet  to  try  to  move  the  programs  forward.  As  it 
turned  out,  we  were  able  to  get  partial  funding  restored,  and 
the  teams'  interim  work  was  not  in  vain. 

In  general,  the  process  has  greatly  facilitated  transitions  to 
Joint  Medical  Advanced  Development  Programs,  even  while 
higher-level  processes  were  being  refined  among  DHA  and  the 
Services.  The  programs  leverage  DHP  and  Service  resources 
and  complement  existing  Service  development  programs. 

The  experience  of  our  JTWGs  reinforces  the  importance  of 
developing  transition  plans  and  joint  multidisciplinary  teams 
for  each  program.  Transition  planning  should  be  initiated  as 
early  as  possible  at  the  portfolio  level,  as  an  S&T  management 
function  to  ensure  that  the  leading  technologies  are  positioned 
properly  for  transition.  In  our  experience,  the  portfolio-level 
perspective  has  enabled  identification  of  top-level  prioritiza¬ 
tions  and  available  funds  from  DHP  and  across  Services.  This 
portfolio  approach  facilitates  leveraging  and  allows  the  tar¬ 
geting  of  resources  toward  the  highest-priority  programs  that 
need  to  transition.  These  processes  may  be  useful  for  other 
medical  or  nonmedical  development  programs.  & 

The  authors  can  be  contacted  through  anthony.e.pusateri.civ@mail.mil. 
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The  Myth  of  Data  Rights 

Peter  A.  Czech 


The  mystery  surrounding  data  rights  stems  from  the  fact  that  intel¬ 
lectual  property  (IP)  and  data  rights  are  among  the  more  complex 
issues  in  acquisition  management. 

How  does  one  reduce  the  complexity  and  make  the  subject  more  understand¬ 
able?  First,  we  must  understand  and  answer  the  following:  What  are  data  rights? 
How  can  we  remove  some  of  the  myths  surrounding  them?  What  can  we  do  if  we  don't 
own  the  data  rights? 


Before  we  answer  these  questions,  a  review  of  history  is  in  order.  Back  in  the  1950s  and 
1960s,  the  U.S.  Government  was  leading  in  research  and  development  (R&D)  spending. 
The  government  spent  between  60  percent  and  70  percent  of  the  national  R&D  expen¬ 
ditures.  Most  of  these  expenditures  focused  on  landing  a  man  on  the  moon.  President 
Kennedy  stated  that  "this  nation  should  commit  itself  to  achieving  the  goal,  before  this 
decade  is  out,  of  landing  a  man  on  the  moon  and  returning  him  safely  to  Earth."  In  those 
days,  the  U.S.  Government  held  all  of  the  IP  cards.  The  government  owned  a  majority  of 
the  IP  on  space  exploration  and  was  able  to  use  it  as  was  required.  After  the  nation  landed 
a  man  on  the  moon,  the  national  R&D  spending  gradually  was  reduced. 


As  we  fast  forward  to  the  year  2000  and  through  to  the  present,  the  R&D  roles  have  re¬ 
versed.  Industry  is  leading  in  R&D  expenditures.  And  industry's  view  of  IP  is  diametrically 
opposed  to  that  of  government.  (See  Figure  1.) 

Industry  View 

IP  is  the  lifeblood  of  a  contractors'  business.  For  most  businesses,  IP  is  the  business.  Their 
IP  allows  them  to  design,  build,  test  and  field  items  that  are  unique  from  their  competi¬ 
tors'  and  allows  customers  to  assign  high  value  to  their  goods  and  services.  Since  I P  is  the 
lifeblood  of  the  contractors'  business,  they  will  protect  it  at  all  costs. 


The  business  IP  allows  it  to  build  wealth  around  the  design,  manufacture,  test,  sustain¬ 
ment  and  disposal  of  an  item.  Each  of  these  phases  of  the  product  development  cycle  is 
a  renewed  income  stream  for  the  business.  Any  business  would  be  loath  to  give  up  even 
one  of  these  income  streams  for  any  reason.  Additionally,  having  the  IP  blocks  a  competi¬ 
tor  from  entering  the  marketplace. 


Czech  is  a  Professor  of  Program  Management  in  the  School  of  Program  Management  of  the  Defense  Systems 
Management  College  at  the  Defense  Acquisition  University,  Fort  Belvoir,  Virginia.  He  previously  worked  for 
Chrysler  Corp. 
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Figure  1.  National  R&D  Expenditures,  by  Funding  Sources 


Note:  “Other"  includes  universities  and  colleges,  state  and  local  governments,  and  other 
nonprofit  organizations. 

Source:  National  Science  Foundation ,  National  Center  for  Science  and  Engineering  Statistics , 
National  Patterns  of  R&D  Resources  (annual  series). 


One  must  remember,  however,  that  all  IP  is  not  codified.  This 
knowledge— or  IP— manifests  itself  in  tribal  knowledge  or  on 
processes  that  a  company  fine-tunes  over  time.  This  informa¬ 
tion  is  not  written  down;  it's  just  how  we  do  things  here.  The 
company  may  not  know  why  the  process  works— but  it  does 
work.  Therefore,  a  third  party  (i.e.,  the  U.S.  Government)  may 
have  the  IP  from  a  company  but  not  the  "secret  sauce"  needed 
to  recompete  a  project. 

Government  View 

The  government  looks  at  I P  from  two  different  lenses.  First,  as 
a  purchaser  of  goods,  government  wants  competition— for  this 
lowers  prices  for  the  goods  and  services  it  buys.  Competition 
also  drives  better  solutions  into  the  marketplace,  and  this  also 
brings  lower  costs. 

The  government  wants  to  receive  the  best  return  on  its  invest¬ 
ment.  The  government  does  not  want  to  pay  more  than  once 
for  the  same  thing.  It  also  doesn't  want  to  be  locked  into  a  sole- 
source  situation,  which  would  enable  the  source  company  to 
charge  higher  prices  than  would  be  possible  in  a  competitive 
situation.  The  government  wants  an  assured  service,  repair 
and  a  modification  source  that  will  provide  best  value  at  an 
affordable  cost. 

The  second  lens  the  government  looks  through  on  IP  begins 
with  our  nation's  Founding  Documents.  Article  I,  Section  8  of 
the  United  States  Constitution ,  provides  that,  "The  Congress 
shall  have  power ...  To  promote  the  Progress  of  Science  and 


useful  Arts,  by  securing  for  lim¬ 
ited  Times  to  Authors  and  Inven¬ 
tors  the  exclusive  Right  to  their 
respective  Writings  and  Discov¬ 
eries."  Thus,  the  government 
wants  private-sector  industries 
to  maintain  their  IP.  This  view 
can  conflict  with  having  compe¬ 
tition.  On  one  hand,  we  want  the 
best  value  for  the  dollar,  and  on 
the  other  hand  we  want  the  best 
outcome  or  product  for  the  pub¬ 
lic  sector.  Where  does  the  myth 
of  data  rights  originate? 

Myths 

The  Merri am -Webster  Diction¬ 
ary  defines  myth  as  "an  idea  or 
story  that  is  believed  by  many 
people  but  that  is  not  true." 
The  first  myth  that  most  folks 
have  is  that  data  rights  and  IP 
are  interchangeable.  They  are 
not!  "Intellectual  Property  is  an 
expression  of  a  new  and  useful 
concept  that  can  be  legally  pro¬ 
tected  such  that  the  originator 
is  granted  certain  exclusive  rights."  However,  data  rights  are 
a  shorthand  way  of  referring  to  the  license  rights  that  the 
Department  of  Defense  (DoD)  acquires  to  use  and  publish 
information.  This  concept  is  the  hardest  to  fathom.  If  we  paid 
for  the  IP,  then  we  should  get  the  IP.  Not  so  fast:  We  may  have 
thought  (or  implied)  that  we  paid  for  the  IP,  under  Federal 
Acquisition  Regulation  (FAR)  27.403,  but  what  was  stated  in 
the  contract?  The  FAR  provides  that: 

"Data  rights  clauses  do  not  specify  the  type,  quantity  of  data 
that  is  to  be  delivered,  but  only  the  respective  rights  of  the  gov¬ 
ernment  and  the  contractor  regarding  the  use,  disclosure  or 
reproduction  of  the  data.  Accordingly,  the  contract  shall  specify 
the  data  to  be  delivered." 

In  other  words  you  only  possess  the  data  that  are  explicitly 
required  in  the  contract.  Use  caution  on  what  you  ask  for  and 
why  you  require  the  information.  That  "why"  must  be  included 
in  your  IP  strategy.  Your  IP  strategy  must  include  the  reasons 
you  require  the  information  and  when  you  need  it.  Remember, 
timing  is  important  in  your  IP  strategy. 

A  delay  in  needing  the  information  may  reduce  the  cost  of 
that  information.  For  example,  if  you  need  the  information  15 
years  from  now,  this  information  will  be  less  valuable  to  the 
contractor  than  it  is  today  and  thus  less  costly  to  you.  Knowing 
that  the  information  in  the  future  will  be  less  valuable  to  the 
contractor  may  allow  you  to  use  an  option  strategy  to  acquire 
the  IP  when  you  really  need  it.  You  could  use  an  option  strategy 
when  you  are  acquiring  data  rights.  Some  questions  to  explore: 
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■  Do  I  need  the  information  to  recompete  the  source? 

When  will  I  recompete? 

■  Do  I  need  the  information  for  diminishing  manufacturing 
sources  of  supply?  When  will  those  sources  be  available? 

Listed  below  are  the  fundamental  questions  that  must  be  an¬ 
swered  by  the  program  team  regarding  its  IP  strategy. 

■  What  IP  do  I  need  for  my  program? 

■  What  rights  do  I  have? 

■  When  do  I  need  this  data? 

■  How  am  I  at  risk? 

■  At  what  price? 

These  questions  need  to  be  answered  for  both  product  defi¬ 
nition  data  and  product  operational  data.  Product  definition 


"By  law,  any  enforceable  right  to  see,  access,  or  have  a  copy 
of  data  requires  an  OMB  [Office  of  Management  and  Budget] 
approved  DID  [Data  Item  Description]  o r  FAR/DFARS  [Defense 
Federal  Acquisition  Regulation  Supplement]  Clause. ...  Therefore, 
DoD  cannot  assume  it  has  any  usable  rights  in  data  that  is  in¬ 
formally  provided  unless  such  rights  are  explicitly  granted  by 
the  contractor  and  reviewed  by  legal  counsel.  All  data  access 
provisions  must  be  reviewed  by  counsel  and  the  data  rights 
in  accessed  information  must  be  addressed  in  the  contract." 

These  takeaways  are  as  important  as  the  data  the  contractor 
shared  with  me  informally— they  don't  give  me  the  right  to 
share  the  information  with  others  (data  rights).  I  only  have  the 
data  rights  if  the  contract  explicatively  grants  that  ownership. 
This  can  get  program  offices  into  some  sticky  situations,  and 
has  done  so. 
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What  if  the  previous  program  managers 
were  focused  on  items  other  than  IP  and  data 
rights?  Is  all  lost,  and  are  you  therefore  stuck 
with  the  contractor  you  have?  No!  Techniques 
are  available  to  foster  competition  without 
possessing  exclusive  data  rights. 


data  encompass  the  drawings  and  specifications  of  the  hard¬ 
ware  and  software  while  product  operations  data  address 
the  maintenance  and  operational  information.  Taking  each 
of  the  above  questions  in  turn  will  allow  us  to  map  out  our 
IP  strategy. 

The  second  myth  regarding  data  rights:  Most  people  interpret 
data  rights  as  IP  ownership.  That  is  not  correct.  Data  rights 
only  apply  to  the  last  part  of  the  definition  provided  by  the  FAR: 

"The  respective  rights  of  the  government  and  the  contractor 
regarding  the  use,  disclosure  or  reproduction  of  the  data  ...  to 
be  delivered." 

You  can't  publish  what  you  don't  have.  What  you  have  may 
be  misleading  or  incorrect.  In  other  words,  you  may  have  the 
right  to  publish  the  information— but  is  it  the  right  information? 
If  you  expect  to  manufacture  the  component  but  have  only 
design  data,  you  may  not  be  able  to  do  so. 

That  brings  us  to  our  third  myth,  which  is  that  the  require¬ 
ments  for  data  delivered  must  be  addressed  in  contract  terms. 
The  government  should  not  require  data  that  are  not  neces¬ 
sary  to  meet  its  needs.  In  the  Army  Materiel  Command's  Army 
Guide  for  the  Preparation  of  a  Program's  Product  Data  Manage¬ 
ment  Strategy  (DMS),  we  find: 


What  if  the  previous  program  managers  were  focused  on 
items  other  than  IP  and  data  rights?  Is  all  lost,  and  are  you 
therefore  stuck  with  the  contractor  you  have?  No!  Techniques 
are  available  to  foster  competition  without  possessing  exclu¬ 
sive  data  rights.  Listed  below  are  a  few  key  strategies  that  can 
be  applied  to  assist  the  program  office  with  their  competitions. 

■  Competitive  Copying 

■  Form-Fit-Function 

■  Direct  Licensing 

■  Leader-Follower 

■  Specific  Acquisition 

■  Reverse  Engineering 

Competitive  Techniques 

In  a  2010  Government  Accountability  Office  report,  the  fol¬ 
lowing  reasons  were  provided  for  the  program  being  stuck  in 
a  sole-source  situation:  "Most  of  the  contracting  and  program 
officials  at  DoD  that  we  spoke  with  pointed  to  the  lack  of  ac¬ 
cess  to  technical  data  as  one  of  the  main  barriers  to  competi¬ 
tion.  Some  of  the  contracting  officers  described  this  condi¬ 
tion  as  essentially  being  'stuck'  with  a  certain  contractor.  ... 
Some  contracting  and  program  officials  have  inquired  about 
the  cost  of  obtaining  the  technical  data,  only  to  discover  that 
the  package  is  not  for  sale  or  that  the  purchase  of  it  would  be 
cost-prohibitive,  especially  the  systems  and  equipment  that 
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have  been  contracted  out  for  decades."  (GAO  Report  10-833). 
This  information  could  dishearten  the  government  program 
manager.  However,  if  you  open  the  aperture,  there  are  options 
for  fostering  competition  without  using  data  rights.  The  gov¬ 
ernment  program  manager  has  many  options  to  explore,  some 
of  them  listed  below.  Examples  of  competitive  techniques  or 
common  methods  of  obtaining  competition  are  drawn  from 
The  Government  Contracts  Reference  Book— A  Comprehensive 
Guide  to  the  Language  of  Procurement  by  R.C.  Nash,  et  al.: 

Competitive  Copying 

One  of  the  most  common  methods  of  obtaining  competition 
of  relatively  simple  items  is  to  solicit  bids  without  furnishing 
technical  data  package.  This  strategy  is  very  commonly  used 
in  the  automotive  industry.  Go  into  any  auto  parts  store  and 
request  a  part  for  your  car.  It  is  a  good  bet  that  the  manufac¬ 
turer  of  the  part  did  not  have  a  data  package  from  the  original 
equipment  manufacturer.  Some  manufacturer  decided  that  it 
could  produce  at  a  lower  cost  a  similar  part  that  would  fit  and 
perform  like  the  original  part.  The  second  manufacturer  found 
what  worked  for  the  part  and  made  improvements  to  produce 
the  component  at  a  lower  cost,  or  with  higher  quality,  or  both. 

Form-Fit-Function 

Base  the  procurement  of  performance  or  functional  specifica¬ 
tions  rather  than  provide  detailed  information  to  meet  govern¬ 
ment  requirements.  Again,  without  proprietary  information, 
I  can  specify  the  form,  fit  and  function  of  a  part  or  system 
and  have  that  system  built.  To  implement  this  strategy,  the 
program  office  will  need  to  supply  the  form,  fit  and  function 
parameters.  This  is  an  exceptional  strategy  when  the  program 
has  an  Open  Architecture  strategy. 

Direct  Licensing 

This  is  an  agreement  between  the  government  and  a  develop¬ 
ment  contractor  that  permits  the  government  to  select  a  sec¬ 
ond  source  after  completion  of  development  and  requires  the 
contractor  to  provide  technical  data  and  technical  assistance 
to  that  contractor  to  make  sure  that  the  product  is  manufac¬ 
tured  completely.  In  exchange,  the  development  contractor  is 
paid  some  combination  of  the  costs  incurred  in  transferring 
the  technology  and  some  sort  of  royalty. 

This  technique  is  applicable  when  the  original  manufacturer  is 
updating  its  business  strategy.  It  may  see  this  business  strat¬ 
egy  as  unprofitable  or  the  state  of  the  art  as  evolving,  and 
therefore  it  is  willing  to  extend  the  profitability  of  its  IP  longer 
by  having  another  manufacturer  bear  the  load  while  reaping 
royalties. 

Leader  Follower 

This  technique  also  establishes  a  direct  relationship  between 
the  original  developer  of  the  item  and  competitors,  but  it  does 
not  call  for  a  royalty  payment.  This  operates  as  procurement 
techniques  under  which  the  sole-source  producer  furnishes 
manufacturing  assistance  and  know-how  to  enable  a  follower 
company  to  become  a  source  for  the  item. 


The  Navy's  shipbuilding  activities  follow  this  strategy.  There  is 
limited  competition  for  building  ships.  Therefore,  the  two  ship¬ 
builders  share  information  so  each  shipyard  can  stay  in  busi¬ 
ness.  The  government  benefits  because  there  is  competition, 
and  the  national  shipbuilding  assets  continue.  The  companies 
benefit  by  staying  in  business. 

Specific  Acquisition 

Another  technique  in  establishing  competition  is  to  purchase 
the  proprietary  rights  from  the  developer  and  then  use  the  data 
as  the  basis  for  a  competitive  procurement.  This  is  the  mindset 
traditionally  used  by  most  program  offices.  They  will  buy  the 
weapon-system  IP  and  use  it  to  have  another  manufacturer 
build  the  system  at  lower  cost.  While  it  sounds  good,  nowhere 
in  the  IP  can  we  find  the  "secret  sauce"  that  will  allow  duplica¬ 
tion  of  the  needed  parts  or  programs. 

For  a  specific  acquisition,  you  must  target  specifically  the 
areas  you  need  to  duplicate.  Understand  the  targets  early, 
and  understand  that  you  will  need  to  have  internal  processes 
to  ensure  that  the  data  are  up  to  date.  The  remaining  action 
is  to  determine  at  what  price  the  contractor  will  sell  the  rights 
to  its  information.  The  difficulty  arises  in  setting  the  price  for 
those  rights. 

Reverse  Engineering 

A  final  method  of  obtaining  competition  without  violat¬ 
ing  proprietary  rights  is  for  the  government  to  "reverse 
engineer"  the  product  and  use  the  drawings  created  in  a 
competitive  environment.  Reverse  engineering  can  do  two 
things.  It  can  enable  you  to  obtain  additional  competitors  in¬ 
stead  of  being  locked  into  one  vendor.  Also,  through  reverse 
engineering,  the  system  or  component  can  be  improved 
over  the  original  design. 

Conclusion 

Data  rights  and  IP  are  complex  subjects.  This  complexity  is 
driven  by  three  sources:  the  government,  industry  and  the 
myths  that  surround  them.  The  contractor  looks  at  the  sub¬ 
ject  through  one  lens.  The  government  looks  at  the  subject 
through  a  different  lens,  and  the  myths  further  exacerbate  a 
complicated  situation. 

We  have  found  that  data  rights  and  IP  are  not  interchangeable. 
The  mere  possession  of  data  rights  doesn't  mean  you  have 
information  that  would  prove  useful  to  create  additional  com¬ 
petition.  Some  other  debunked  mysteries  include  the  question 
of  whether  I  can  make  separate  use  of  the  data  that  I  have. 
Legally,  you  can  use  the  information  only  if  the  information  is 
identified  in  the  contract. 

Even  if  we  don't  own  the  IP  or  the  data  rights,  all  is  not  lost. 
There  are  competitive  techniques  that  a  program  office  can  ex¬ 
plore  to  help  foster  competition  and  lower  the  costs  of  weapon 
systems  acquisition. 


The  author  can  be  contacted  at  peter.czech@dau.mil. 
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"Exporting"  Defense  AT&L 
Has  Far-Reaching  Impact 


II 


The  early  bird  catches  the  worm."  Who  doesn't  know  this  proverb?  A  study  club  in 
the  Republic  of  Korea  (ROK)  defense  acquisition  agency  was  born  and  has  grown 
with  this  phrase  as  the  key  tenet  of  its  effort  to  improve  defense  acquisition  profes¬ 
sionalism  by  studying  the  Defense  AT&L  magazine  published  for  the  U.S.  Depart¬ 
ment  of  Defense  (DoD)  by  the  Defense  Acquisition  University  (DAU).  The  study 
club's  name  originated  from  that  tenet  as  the  "Early  Birds  Study"— or  the  EBS,  for  short.  This  year, 
the  EBS  will  celebrate  the  10th  anniversary  of  its  foundation  and  is  almost  as  old  as  the  South 
Korean  Government  organization  of  which  it  is  part. 


Background  and  Context 

The  South  Korean  Government— from  the  National  Assembly  to  the  Ministry  of  National  Defense  (MND),  and  the 
Defense  Acquisition  Program  Administration  (DAPA),  which  is  the  equivalent  of  the  DoD's  Office  of  Acquisition, 
Technology,  and  Logistics  (AT&L)— has  been  working  toward  a  more  responsive,  efficient  and  transparent  acquisi¬ 
tion  system  that  meets  evolving  defense  needs  while  developing  a  more  professional  acquisition  workforce.  Well, 
that  sure  sounds  easy,  we  all  might  say,  but  it  is  a  daunting  challenge  to  assess  and  find  effective  ways  to  make 
the  fundamental  changes  needed  while  staying  true  to  a  strong  bureaucratic  culture  that  has  a  long  track  record 
and  some  amazing  successes.  In  an  increasingly  global  acquisition  environment,  Korean  officials  recognize  that 


Tripp  is  director  of  international  programs  at  the  Defense  Acquisition  University's  Defense  Systems  Management  College  at  Fort  Belvoir, 
Virginia.  Hwa  Yu  is  chief  of  the  liaison  office  for  the  Joint  U.S.  Military  Affairs  Group-Korea  as  part  of  the  U.S.  Embassy  in  Seoul ,  South  Korea , 
liaising  with  the  Korean  Defense  Acquisition  Program  Administration. 
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Publications  of  the  Early  Birds  Study  group  draw  information  and  inspiration  from  Defense  AT&L  magazine. 

Collage  of  Early  Birds  Study  Group  photos. 


change  is  necessary  to  survive  and  prosper.  The  lessons  are 
certainly  clear  to  the  young  intellectual  professionals  who  will 
form  the  government  of  the  future.  This  article  shines  a  brief 
spotlight  on  just  such  a  group. 

DAPA 

To  talk  about  the  EBS,  it  is  first  necessary  to  understand  the 
Korean  Defense  Acquisition  Program  Administration  (DAPA). 
The  DAPA  was  launched  on  Jan.  1, 2006,  and  consolidated  all 
acquisition  functions  scattered  across  the  MND,  military  Ser¬ 
vices  and  defense  agencies.  Its  primary  missions  are  acquiring 
weapon  systems  for  Korea's  military  forces  and  strengthen¬ 
ing  the  Korean  defense  industry.  Many  military  officers  previ¬ 
ously  in  charge  of  acquisition  programs  were  transferred  to  the 
DAPA  with  the  newly  designated  specialty  of  "acquisition," 
and  yet  many  were  new  to  the  concept  of  defense  acquisition. 

Joint  Military  Affairs  Group 

Another  key  organization  that  should  be  introduced  is 
the  Joint  U.S.  Military  Affairs  Group-Korea  (JUSMAG-K). 
JUSMAG-K  is  a  security  cooperation  organization  work¬ 
ing  in  Korea  to  support  the  alliance  by  advancing  U.S.-ROK 


mutual  security  interests.  JUSMAG-K  closely  works  with 
the  MND,  military  Services  and  the  DAPA,  particularly  on 
foreign  military  sales  (FMS)  and  defense  cooperation  in 
armaments  programs.  JUSMAG-K  has  maintained  liaison 
offices  since  the  MND's  establishment. 

The  Beginning  of  the  EBS 

With  establishment  of  the  DAPA,  the  chief  of  JUSMAG-K  reas¬ 
signed  Hwa  Yu,  who  had  worked  at  the  MND  Liaison  Office 
for  the  several  years,  to  the  DAPA  Liaison  Office  as  the  DAPA 
Liaison.  She  had  just  completed  a  master's  course  in  Defense 
Project  Management  at  the  Korean  National  Defense  Univer¬ 
sity.  Yu  was  new  to  the  job  and  felt  the  need  to  gain  acquisition 
knowledge  on  both  ROK  and  DoD  defense  acquisition  and  to 
develop  professional  relationships  with  DAPA  personnel. 

Yu  met  then-Maj.  Taijun  Park,  whom  she  had  known  dur¬ 
ing  cooperation  on  the  U.S. -Korea  Multiple  Launch  Rocket 
System  (MLRS)  project  and  who  was  now  assigned  to  the 
DAPA.  They  talked  about  regularly  practicing  English  and 
studying  defense  acquisition  and  agreed  to  form  a  club  to 
study  DoD  acquisition  in  English.  In  the  first  week  of  April 
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2006,  Yu,  Park,  Maj.  Hansoo  Park  and  Lt.  Col.  Hyeok  Seo 
gathered  at  the  conference  room  of  the  JUSMAG-K  liaison 
office  for  the  first  time  at  7  a.m.  The  DAPA  compound  was 
still  quiet  before  the  official  government  workday  began  at 
9  a.m.  Because  they  recognized  they  had  developed  strong 


perspectives.  Invited  guests  include  acquisition  leaders,  such 
as  a  former  Minister  of  Defense  for  Acquisition  who  offered 
strategic  thinking  and  a  vision  of  the  way  forward  in  defense 
acquisition.  The  group  also  has  hosted  a  legal  advisor  from 
a  well-known  law  firm  with  extensive  career  knowledge  and 


As  the  club  members  studied 
trends  in  defense  acquisition,  they 
noted  the  importance  and  impact 
of  technology  protection. 


momentum,  the  group  registered  in  the  DAPA  as  a  "Commu¬ 
nity  of  Practice  (CoP)"  on  May  3, 2006.  Park  first  came  up  with 
"Early  Birds"  for  the  study  club  name  and  Yu  proposed  EBS, 
an  abbreviated  version  of  the  Early  Birds  Study,  which  became 
the  official  study  club  name. 

The  club  established  two  main  objectives:  (1)  study  the  De¬ 
fense  AT&L  magazine  published  in  English  to  learn  advanced 
defense  acquisition  systems,  compare  these  with  the  ROK 
systems  and  find  ways  to  improve;  (2)  and  share  what  the 
club  members  study  with  the  rest  of  the  DAPA  personnel. 
Those  two  objectives  were  reflected  in  the  EBS  charter  and 
have  remained  intact  since. 

Activities 

Regular  Meeting:  Based  on  the  two  objectives,  the  EBS 
members  meet  twice  weekly  to  study  defense  acquisition. 
The  members  take  turns  leading  the  weekly  study  sessions  in 
an  agreed  sequence.  The  lead  member  must  choose  a  topic, 
primarily  from  the  Defense  AT&L  magazine,  translate  it  into 
Korean  and  develop  a  study  plan.  Government  Accountability 
Office  (GAO)  reports,  Defense  Acquisition  Research  Journal 
papers,  and  other  important  reports  such  as  Defense  Ac¬ 
quisition  Performance  Assessment  reports  also  have  been 
selected  for  discussion. 

Friday  Social  Meetings:  These  primarily  are  lunch  gath¬ 
erings  to  strengthen  the  personal  relationships  within  the 
group.  While  regular  meetings  are  required,  the  Friday  lunch 
is  optional. 

Seminars:  To  achieve  the  knowledge-sharing  goal  of  the  EBS, 
the  club  holds  quarterly  seminars  within  DAPA  and  outside 
DAPA  with  interested  acquisition  personnel. 

Sessions  With  Experts:  The  EBS  Club  often  invites  defense 
acquisition  experts  to  regular  sessions,  seminars  or  social 
meetings  so  that  the  members  can  gain  in-depth  knowledge 
and  share  insights  from  subject-matter  experts  and  broader 


experience  in  defense  contracting  as  well  as  a  cost  analyst  and 
a  3-D  printing  company  chief  executive  officer. 

Annual  Publication  of  Study  Materials:  Each  year,  the  EBS 
assembles  the  Korean  translations  of  articles  they  had  stud¬ 
ied  for  the  past  year.  Currently,  the  members  are  reviewing 
the  translated  articles  for  the  10th  volume  of  their  study  re¬ 
sults.  But  the  study  results  book  provides  more  than  trans¬ 
lations.  Some  topics  of  interest  for  DAPA,  like  Acquisition 
Strategy,  that  were  studied  more  in  depth  resulted  in  the 
group  proposing  certain  changes.  Additionally,  the  DoD  deci¬ 
sion-making  and  contracting  systems  and  other  useful  infor¬ 
mation  have  been  added  to  the  books  as  appendices,  in  order 
to  help  DAPA  readers  better  understand  the  DoD  systems. 
The  EBS  books  are  used  as  sources  of  information  on  DoD 
acquisition  policies  and  systems  and  trends  and  to  provide 
ideas  for  improvement  to  DAPA  officials.  The  selected  topics 
usually  are  of  DAPA  interest,  and  the  EBS  books  provide  easy 
access  in  the  Korean  language  to  current  and  timely  informa¬ 
tion  that  otherwise  would  be  difficult  to  acquire. 

Membership 

The  EBS  members  include  both  military  and  civilian  officials 
assigned  to  the  DAPA.  English  language  ability  is  a  require¬ 
ment  of  those  who  join  the  EBS.  Every  member  is  expected 
to  actively  participate  in  the  study  session.  There  is  a  2-week 
probationary  period  in  order  to  provide  an  opportunity  to  see 
if  a  prospective  member  can  meet  EBS'  expectations. 

The  17  current  members  have  a  variety  of  working  back¬ 
grounds  and  include  active-duty  military  (five  Army,  four 
Navy,  and  two  Air  Force  members),  five  government  civilian 
employees  (including  a  senior  executive  official)  and  a  U.S. 
Embassy  employee. 

Achievements 

Exposing  DAPA  personnel  to  the  latest  DoD  acquisition  poli¬ 
cies  and  practices  through  books,  Internet  postings,  semi¬ 
nars,  etc.,  has  been  well  received  by  many  DAPA  officials 
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An  Early  Birds  Study  session  in  April  2014. 

Early  Birds  Study  Group  Photo. 


for  providing  "food  for  professional  thought."  A  few  tangible 
contributions  follow: 

Since  December  2014,  the  EBS  has  held  three  seminars  in 
the  DAPA  with  the  themes  that  are  of  great  interest  to  DAPA 
employees— such  as  professionalism,  acquisition  ethics  and 
leadership.  EBS  introduced  DoD  policies  and  practices  from 
sources  such  as  the  Better  Buying  Power  initiatives  (based 
on  a  BBP  3.0  White  Paper  by  Under  Secretary  of  Defense  for 
AT&L  Frank  Kendall);  "Ethics  and  Acquisition  Professional¬ 
ism:  It's  All  about  Trust,"  an  article  by  Kendall  in  the  Sep- 
tember-October  2014  Defense  AT&L;  and  "The  21st-Century 
Acquisition  Leader,"  an  article  by  Paul  E.  Turner  in  the  Janu- 
ary-February  2015  Defense  AT&L. 

Comparisons  with  DAPA  systems  were  followed  by  exten¬ 
sive  discussions  at  each  seminar.  The  club  also  has  held 
several  joint  seminars  with  the  ROK  Army  HQs  and  the 
Defense  Agency  for  Quality  and  Technology  to  introduce 
DoD  and  ROK  acquisition  systems  to  the  requirement  and 
technology  planning  communities  and  to  hear  their  opin¬ 
ions.  These  seminars  have  helped  the  participants  to  better 
understand  both  their  own  and  U.S.  acquisition  systems. 
A  large  portion  of  the  ROK's  offshore  procurement  comes 
from  the  United  States.  The  EBS  club  members  aspire  to 
expand  their  reach  to  the  other  military  Services  and  the 
Joint  Chiefs  of  Staff  to  invigorate  communications  within 
the  requirements  community. 

Prior  to  the  2013  and  2014  U.S. -ROK  Defense  Technological 
and  Industrial  Cooperation  Committee  (DTICC)  meetings  co¬ 
chaired  by  the  U.S.  Under  Secretary  of  Defense  (AT&L)  and 
the  Korean  Minister  of  Defense  Acquisition,  the  club  members 
prepared  the  Korean  delegation  on  topics  and  presented  ideas 


for  discussion  points.  One  of  the  topics  related  to  the  DoD's 
Better  Buying  Power  initiatives.  The  club  also  held  a  seminar 
on  the  BBP  initiatives,  sharing  the  latest  DoD  trends  that  the 
ROK  is  trying  to  emulate,  and  discussed  the  direction  in  which 
the  DAPA  should  evolve. 

The  EBS  has  made  significant  contributions  to  formulating 
defense  technology  security  and  export  control  policies.  As 
the  club  members  studied  trends  in  defense  acquisition,  they 
noted  the  importance  and  impact  of  technology  protection  and 
subsequently  studied  U.S.  technology  security  and  export  con¬ 
trol  systems  and  shared  the  knowledge  through  seminars  and 
lectures  at  the  DAPA  training  center.  After  Yu  took  the  DAU's 
PMT  203  course  on  International  Security  and  Technology 
Transfer/Control  in  November  2011,  the  club  put  together  de¬ 
tailed  information  on  defense  technology  security  and  export 
control  supporting  the  establishment  of  a  defense  technology 
security  office.  That  effort  is  about  to  bear  more  fruit  in  the 
form  of  a  proposed  defense  technology  security  law,  which  is 
currently  under  review  in  the  ROK  National  Assembly. 

The  best  achievement,  perhaps,  is  the  professional  growth  of 
the  members  of  the  EBS.  The  members  evolve  from  acquisition 
professionals  to  acquisition  leaders  themselves.  EBS  President 
Col.  Ffyeok  Seo  of  the  ROK  Army  led  the  nascent  EBS  club  as 
a  freshly  promoted  lieutenant  colonel  and  now  is  in  charge  of 
Korea's  important  tank  projects.  Acting  as  a  servant  leader 
to  the  group  members  (see  above-mentioned  "21st  Century 
Acquisition  Leader"  Turner  article  in  Defense  AT&L),  he  has 
endeavored  to  extend  the  EBS  members'  potential  to  become 
global  acquisition  leaders.  Recently,  he  participated  in  the  In¬ 
ternational  Defense  Educational  and  Acquisition  Arrangement 
(IDEAA)  seminar  in  the  United  Kingdom,  where  he  briefed  the 
international  audience  of  acquisition  and  educational  experts 
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The  EBS  is  quite  well  known  in  the  DAPA 
since  the  group  has  won  the  first  place  in 
the  DAPArs  annual  CoP  competitions  almost 
every  year,  ranking  second  just  a  few  times. 


on  Korean  approaches  to  acquisition  challenges  and  the  need 
for  training  and  education.  It  was  the  realization  of  one  of  the 
dreams  of  the  EBS.  The  U.S.  delegation  was  led  by  Katharina 
McFarland,  Assistant  Secretary  of  Defense  for  Acquisition, 
and  James  Woolsey,  President  of  DAU. 

Seo  currently  is  working  on  the  HR  development  of  the  DAPA 
by  actively  proposing  training  and  career  development  mod¬ 
els  and  improvement  measures  for  the  Korean  acquisition 
professionals.  Lt.  Col.  Hansoo  Park  is  back  in  the  EBS  after 
his  return  from  a  3-year  overseas  assignment  for  which  he 
was  personally  selected  as  the  Army  Program  Officer  at  the 
Korean  Embassy  in  Washington  D.C.  He  says  the  knowledge 
and  understanding  of  the  DoD  acquisition  system  he  had 
gained  through  the  EBS  was,  without  question,  extremely 
helpful  to  the  performance  of  his  duty  in  the  United  States. 
Now  as  the  "early  bird  continues  to  fly,"  he  is  sharing  his 
experience  and  insight  from  the  United  States  with  junior 
members  of  the  EBS. 

Awards 

The  EBS  is  quite  well  known  in  the  DAPA  since  the  group  has 
won  the  first  place  in  the  DAPA's  annual  CoP  competitions 


almost  every  year,  ranking  second  just  a  few  times.  Lately,  the 
EBS  also  was  recognized  in  the  Top  10  at  the  national  govern¬ 
ment  level  CoP  competition,  in  which  about  100  CoPs  com¬ 
peted.  It  was  the  first  time  any  DAPA  CoP  received  national 
government  recognition. 

Conclusion 

The  Early  Birds  Study  group  is  a  powerful  example  of 
successful  international  outreach  and  collaboration.  It  is 
unique  in  being  completely  driven  from  within— by  a  group 
of  young  Korean  professionals  who  are  making  waves.  The 
U.S.  co-author  of  this  article  has  worked  for  years  in  seek¬ 
ing  improved  and/or  strengthened  international  collabora¬ 
tion  and  rarely  has  encountered  such  an  energetic,  highly 
effective  group  of  acquisition  professionals.  The  growing 
professionalism  in  the  DAPA  workforce  will  pay  acquisition 
outcome  benefits  between  our  two  nations  in  current  and 
future  programs.  Just  remember— it's  the  early  bird  that 
catches  the  worm!!  & 


The  authors  can  be  contacted  at  duane.tripp@dau.mil  and  hwa.yu2.ln@ 
mail.mil. 


Members  of  the  current  Early  Birds  Study  group  hold  copies  of  Defense  AT&L  magazine— a  resource  for  their  work.  Front  Row, 
from  left  to  right:  Hwa  Yu;  Lt.  Col.  Hansoo  Park;  Col.  Hyeok  Seo  (president);  Cmdr.  Yisub  Ahn;  and  Hyoungcheol  Kim.  Back  Row, 
left  to  right:  Lt.  Jeongha  Kim;  Maj.  Sangho  Aum;  Maj.  Joseph  Kim;  Sangeun  Lee;  Yunhee  Do;  Lt.  Cmdr.  Sehoon  Cha;  Seok  Kim. 

Photo  by  Korean  Defense  Acquisition  Program  Administration  Photographer  Jaehoon  Kim. 
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Airborne,  All  The  Way,  3D  From  the  Sky 

Approach  for  an  Air  Drop  Capable,  Medium  Tactical  Vehicle 

CPT  Peter  F.  Syverson,  USA 


In  warfighting  today,  the  U.S.  mili¬ 
tary  rolls  into  combat  with  heav¬ 
ily  armored  vehicles  capable  of 
protecting  their  occupants  from 
large  and  highly  lethal  improvised 
explosive  device  (I ED)  blasts.  Given 
that  threat,  it  would  seem  surpris¬ 
ing  that  our  forces  still  require  unar¬ 
mored  vehicles  for  combat  missions. 

Product  Manager  Medium  Tactical  Vehicle  (PM  MTV)  fields  three  models  of  unar¬ 
mored  vehicles:  the  Airborne's  MTV  Low  Velocity  Air  Drop  (LVAD)  M1081,  M1093, 
and  M1094  (2.5T  Cargo,  5T  Cargo,  and  5T  Dump  respectively).  These  models  were 
designed  in  the  early  1990s  to  provide  low-tech  solutions  to  an  Airborne  requirement. 
Sustainment  and  production  problems  have  arisen  for  the  LVAD  fleet  in  recent  years. 
The  largest  problem  to  overcome  is  the  obsolescence  of  the  LVAD  cabs.  Replacement 
parts  are  nonexistent  in  many  cases,  and  contractors  are  unwilling  to  produce  the  small 
quantities  required  by  the  Army  to  sustain  the  fleet. 

Given  the  lack  of  replacement  cabs  or  parts,  PM  MTV  has  discovered  that  old  LVADs 
are  potentially  coded  out  due  to  the  unavailability  of  a  door  panel  that  costs  a  few 
hundred  dollars,  even  though  the  rest  of  the  truck  is  operational.  When  the  cab  shell 
was  last  built  in  2008,  it  was  proprietary  to  the  contractor  that  manufactured  it.  At 
the  time,  the  government  did  not  purchase  the  Technical  Data  Package  (TDP)  or  the 
dies  used  to  stamp  out  the  cab.  The  dies  have  since  been  sold  off,  highly  modified 
and  consequently  destroyed.  A  TDP  is  necessary  in  order  for  PM  MTV  to  build  cabs 
or  replacement  panels.  A  reverse  engineering  project  began  in  2012  to  build  a  TDP 
in  conjunction  with  current  prime  contractor.  The  cost  was  well  more  than  $6  million 
to  validate  this  TDP,  build  the  29  tooling  stamps  required  for  all  the  body  panels,  and 
produce  five  prototype  cabs.  It  would  take  more  than  450  days  after  award  to  produce 
the  five  prototype  cabs  (remember  these  numbers).  For  a  niche  population  of  airborne 
trucks  with  no  new  production  on  the  horizon,  the  price  killed  the  project. 

But  within  three  years,  units  in  the  field  wonder  why  they  cannot  acquire  a  cab  part 
from  the  Defense  Logistics  Agency  (DLA).  The  PM  and  DLA  now  face  the  same  di¬ 
lemma  they  faced  a  few  years  earlier.  Trying  to  think  outside  the  box  for  a  low-cost 
solution  to  TDP  validation  and  prototyping,  on  a  small  budget  and  with  a  short  time¬ 
line,  I  remembered  reading  about  the  Navy's  research  in  constructing  3D  buildings. 


Syverson  is  the  Assistant  Product  Manager  for  Medium  Tactical  Vehicle  at  Tank-automotive  and  Arma¬ 
ments  Command  (TACOM).  He  graduated  and  was  commissioned  from  Northern  Illinois  University  and 
has  an  Industrial  Technology  Degree.  His  previous  assignments  included  4th  Brigade  Combat  Team,  82nd 
Airborne  Division  as  Commander  of  both  A  Troop,  4-73  Cavalry  (Airborne)  and  Headquarter  and  Head¬ 
quarters  Company,  l-508th  Parachute  Infantry  Regiment.  He  earlier  was  a  Reconnaissance ,  Surveillance 
and  Target  Acquisition  Platoon  Leader  with  5-1  Cavalry  Regiment ,  1/25  Infantry  Division. 
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Low  Velocity  Air  Drop  rigged  at  Fort  Bragg  Heavy  Drop  Rig  Site  in  North  Carolina. 

Photo  by  the  Army  Product  Management  Office  for  Medium  Tactical  Vehicles. 


I  thought  if  the  Department  of  Defense  (DoD)  can  print  in 
that  size  in  concrete,  why  not  a  6x4-foot  truck  cab?  With  the 
Detroit  Auto  Show  in  town,  I  read  about  a  company  called 
Local  Motors  that  printed  a  3D  car  and  would  exhibit  both  its 
car  and  Local  Motors'  additive  manufacturing  technology  at 
the  show.  I  decided  to  make  the  Detroit  Auto  Show  my  place 
of  duty  in  order  to  research  the  feasibility  of  using  additive 
manufacturing  to  build  a  3D  composite  cab. 

At  the  show,  I  was  impressed  with  Local  Motors'  Strati  3D  car 
and  impressed  again  with  the  technology  partner's  exhibit. 
Oak  Ridge  National  Laboratory  (ORNL)  was  showcasing  its 
3D  printed  Shelby  Cobra  next  to  Local  Motors,  and  all  the 
minds  behind  the  project  were  present  to  answer  questions. 
Located  conveniently  across  from  the  ORNL  exhibit  was  the 
Tank-automotive  and  Armaments  Command  (TACOM) 
exhibit  featuring  an  autonomous  legacy  MTV  with  an  un¬ 
armored  cab.  As  I  spoke  with  the  ORNL  people  who  had 
developed  the  3D  Cobra,  I  pointed  to  our  MTV  and  asked  if 
it  was  feasible  to  print  the  shell  of  the  cab.  To  my  delight,  the 
short  answer  was  "Yes." 


cabs  and  replacement 
parts  to  keep  the  leg¬ 
acy  trucks  in  the  field. 
Since  our  visit,  the  re¬ 
search  staff  at  ORNL 
has  agreed  to  proto¬ 
type  our  cab,  starting 
with  the  most  complex 
part— a  complete  door 
assembly.  If  the  door 
proves  to  be  feasible 
and  functions  properly 
on  an  existing  LVAD 
cab,  PM  MTV  will  move 
forward  with  the  proto¬ 
type  of  the  whole  cab. 
With  the  assistance  of 
the  DoD  Manufacturing 
Technology  Program, 
all  this  will  happen  for 
less  than  $100,000 
and  within  a  few  weeks 
of  the  project's  start. 
Compared  to  the  $6 
million  and  more  than 
450  days  for  the  conventional  manufacturing  method,  3D 
may  end  up  being  the  better  buy. 

Although  this  information  is  not  new  to  some,  I  wanted  to 
discuss  the  many  potential  benefits  of  additive  manufactur¬ 
ing  to  both  the  Army  and  the  PM  when  looking  at  the  LVAD. 
As  we  look  at  cost  as  our  largest  consideration  in  a  physi¬ 
cally  constrained  environment,  we  look  first  and  foremost  at 
the  departure  from  the  traditional  manufacturing  process. 
As  mentioned,  29  stamps  are  required  to  make  the  current 
panels  that  comprise  the  LVAD.  These  stamps,  in  2012  dol¬ 
lars,  cost  more  than  $4  million  and  required  6  months  just  to 
manufacture.  Stamps  also  require  large  machines  or  fixtures  to 
function,  in  addition  to  an  assembly  line  that  requires  time  to 
set  up  and  calibrate.  By  shifting  to  an  additive  manufacturing 
process,  we  fundamentally  shift  away  from  tooling  and  set-up 
time.  Theoretically,  a  company  like  Local  Motors  can  build  their 
Strati  today,  and,  by  tomorrow,  with  a  TDP  and  3D  models, 
produce  LVAD  cabs  on  the  same  manufacturing  floor.  That 
concept  is  revolutionary,  given  how  the  automotive  industry 
traditionally  builds  vehicles. 


Oak  Ridge  National  Laboratory  is  at  the  leading  edge  of  large- 
scale  additive  manufacturing  and  produces  at  speeds  and 
sizes  that  make  printing  a  high-strength  LVAD  cab  very  fea¬ 
sible.  The  lead  PM  MTV  LVAD  engineer,  Jason  Zebrowski, 
and  I  decided  to  take  a  trip  to  visit  ORNL  in  Knoxville,  Ten¬ 
nessee.  Our  idea  was  to  produce  a  3D  composite  prototype 
cab  that  looks  and  functions  the  same  as  the  old.  Ideally, 
PM  MTV  will  achieve  a  validated  TDP  and  manufacturing 
method  that  requires  no  tooling  and  no  production  line  and 
that  has  the  ability  to  build  affordable,  low  volumes  of  new 


The  PM  would  never  reach  thousands  of  units  of  production 
for  LVAD,  and  we  do  not  want  to  do  so.  Low  volume  is  our 
goal,  and  our  dilemma.  Industry  and  the  Army  do  not  want 
to  produce  a  few  hundred  anything,  especially  something 
as  complex  as  a  vehicle  cab.  Due  to  the  time  and  effort  re¬ 
quired,  low  production  runs  leave  little  profit  for  industry 
and  huge  costs  to  the  customer  (although,  for  deep  enough 
pockets,  this  dilemma  does  not  apply).  With  additive  manu¬ 
facturing,  we  sidestep  the  issue.  Given  a  total  fleet  of  2,200 
vehicles,  PM  MTV  would  produce  only  about  900  vehicles 
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(maximum)  to  replace  the  obsolete  20-year-old  nitial  (AO) 
imodels  and  a  limited  run  of  replacement  parts  for  the  AIR 
models  remaining  in  the  fleet  over  the  next  5  years. 

Continuing  to  take  cost  into  consideration,  PM  MTV  has 
looked  at  the  materials  used  in  additive  manufacturing.  At 
$2.40  to  $5.00  per  pound,  the  composites  and  thermoplas¬ 
tics  utilized  today  become  very  attractive  when  analyzing  our 
current,  800-pound  metal  LVAD  cab.  As  we  explored  these 
composites,  MTV  discovered  that  they  also  are  largely— up 
to  80  percent— recyclable.  Recyclable  material  can  have  huge 
implications,  if  the  Army  ever  invests  in  3D  printers  in  its  de- 
pot-level  maintenance  facilities.  Imagine  a  scenario  in  which 
an  LVAD  comes  into  maintenance  with  a  damaged  door  and 
requires  a  new  one.  The  chief  maintenance  officer  takes  off 


mission,  but  the  LVAD  may  reach  a  level  of  ballistic  protection 
not  seen  in  the  legacy  cabs— depending  on  the  composite  we 
choose  and  internal  design  structure  of  the  panels. 

Additionally,  a  persistent  problem  in  the  legacy  LVAD  fleet  is 
corrosion  of  the  earlier  AO  models.  Once  a  cab  is  damaged  and 
the  bare  metal  exposed,  corrosion  rapidly  sets  in  if  not  checked 
by  corrective  painting  or  part  replacement.  A  plastic  compos¬ 
ite  cab  will  help  reduce  the  DoD's  expenditures  on  corrosion 
prevention  programs  and  use  of  the  ever  time-consuming  and 
difficult  Chemical  Agent  Resistent  Coating  paint.  Depending 
on  the  plastic  composite  chosen  for  the  LVAD,  we  may  even 
avoid  painting  altogether.  Some  plastics  are  translucent,  which 
allows  the  adding  of  color  pellets  to  achieve  olive  drab  or  tan 
paint  schemes. 


As  I  spoke  with  the  ORNL  people  who  had  developed  the  3D  Cobra, 

I  pointed  to  our  MTV  and  asked  if  it  was  feasible  to  print  the  shell  of 
the  cab.  To  my  delight,  the  short  answer  was  “Yes." 


the  door,  strips  it,  throws  the  door  through  the  shredder,  then 
in  short  order  prints  a  new  one  for  the  paratrooper,  using  the 
same  material  as  the  old.  All  the  maintenance  officer  needs  is 
a  3D  printer  and  the  3D-model  computer-aided  design  (CAD) 
data  from  DLA. 

I  also  have  been  asked  why  PM  MTV  is  pursuing  the  same 
LVAD  cab  design.  The  answer  is  that  PM  MTV  wants  to 
maintain  all  the  current  National  Stock  Numbers  and  part 
numbers  that  make  up  the  rest  of  the  cab  (seats,  steer¬ 
ing  columns,  hinges,  etc.)  and  avoid  spending  the  time  and 
money  on  engineering  new  when  parts  already  exist.  By 
doing  so,  we  can  create  a  new  stock  of  replacement  panels 
for  the  LVADs  still  in  service,  reduce  logistic  footprints  and 
save  money  by  not  reinventing  a  truck  that  has  a  limited 
production  quantity. 

Performance  is  the  biggest  question  PM  MTV  has  for  our 
prototype.  While  we  believe  ORNL  will  produce  a  composite 
cab  that  looks  and  functions  like  our  legacy  sheet-metal  cab, 
we  will  need  to  conduct  static  and  live  drop  testing  from  an 
aircraft  to  determine  if  it  will  pass  test  requirements.  With 
that  said,  there  are  promising  prospects  that  we  will  succeed, 
given  our  options  of  composites  with  varying  strengths  and 
mechanical  properties.  These  properties  will  have  added 
benefits  that  are  not  possible  for  the  current  metal  cabs. 
For  instance,  LVADs  are  exempt  from  current  MTV  armor 
requirements,  due  to  weight  and  the  nature  of  the  Airborne's 


The  schedule  of  this  ambitious  project  has  raised  some 
eyebrows,  and  we  understand  why.  To  counter  the  skepti¬ 
cism,  I  would  have  to  say  you  need  to  see  ORNL's  additive 
printing  process  in  action.  As  you  discover  where  3D  print¬ 
ing  is  today  and  begin  to  understand  the  process,  you  gain 
an  appreciation  for  how  quickly  a  prototype  can  develop 
and  move  to  production.  While  at  ORNL,  Zebrowski  and  I 
witnessed  an  exchange  between  two  engineers  deliberating 
why  a  design  of  theirs  had  printed  poorly.  In  a  20-second 
exchange,  they  decided  to  modify  the  design  on  the  CAD 
model  and  print  another  one.  That  is  two  prototypes  and 
one  design  change  created  in  a  single  day— and,  for  the 
complexity  of  the  part,  it  was  quite  impressive.  Knowing 
how  fast  the  staff  there  move,  ORNL  thinks  they  can  create 
the  whole  LVAD  cab  in  a  2-  to  3-week  timeline.  This  includes 
consideration  of  design  errors  and  reprints  for  incorporation 
in  a  final  validated  TDP. 

That  is  why  PM  MTV  hopes  to  have  the  cab  prototype  in  hand 
by  summer  and  a  completed  truck  assembled  by  the  end  of 
2015  to  take  to  test  in  2016.  Is  it  ambitious?  Absolutely.  But 
with  additive  manufacturing,  such  a  timeline  becomes  fea¬ 
sible,  and  working  with  limited  resources  under  short  time 
constraints  is  what  (former)  paratroopers  do.  PM  MTV  and 
I  are  up  for  the  challenge  and  look  forward  to  providing  an 
update  in  the  near  future. 


The  author  can  be  contacted  at  peter.f.syverson.mil@mail.mil. 
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WHAT 

YOU  THINK! 


We  like  happy  readers!  That’s  why  we  want  to  know  what  you  think.  Your  feedback  will  ensure 
we  continue  to  produce  a  magazine  that  is  interesting  and  relevant  to  your  job.  Simply  respond 
to  the  questions  below  and  fax  or  email  this  form  to  Defense  AT&L  Magazine.  All  responses  are 
anonymous  and  will  be  used  only  to  improve  Defense  AT&V s  services. 

Please  rate  the  overall  quality  of  the  magazine. 
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Are  there  any  other  comments  you  would  like  to  provide? 


Email:  datlonline@dau.mil 
Fax:  703-805-2917 


Roy  Wood  ■  Patrick  Barker 


wonderful  organ: 
It  starts  working 
when  you  get  up 
in  the  morning, 
and  doesn't  stop 
until  you  get  to 

the  office." 

—Robert  Frost 


The  lump  of  gray  matter  you  call  your  brain  has  vast  comput¬ 
ing  power  that  performs  a  plethora  of  vital  tasks.  It  regu¬ 
lates  your  bodily  functions,  movements  and  emotions.  It 
processes  and  makes  sense  of  incoming  sights,  sounds, 
smells  and  touch  sensations.  The  brain  can  store  years  of 
memories  and  allow  you  to  perform  complex  tasks  such  as  analysis, 
reasoning,  goal  setting  and  planning. 

Yet,  even  with  all  these  capabilities,  the  brain,  paradoxically,  has  some  rather 
stark  limitations.  It  makes  sense  then  that  the  more  we  understand  about  the 
capabilities  and  limitations  of  our  brains,  the  better  we  are  able  to  coax  the  maxi¬ 
mum  productivity  from  our  internal  computers.  The  goal  is  not  necessarily  to 
increase  our  brain's  capacity,  but  gain  more  efficiency  and  effectiveness  from 
the  capacity  we  already  have. 

The  Neocortex  and  the  Limbic  System 

Without  going  too  deeply  into  the  science  of  the  brain,  we  would  point  out  two 
components:  the  neocortex,  located  atop  the  cerebrum's  frontal  lobes  and 
which  is  the  computational  portion  of  the  brain,  and  the  limbic  system  beneath 
the  cerebrum  and  associated  with  long-term  memory  and  emotions.  In  our 


Wood  is  Acting  Vice  President  of  the  Defense  Acquisition  University.  Barker  is  a  Professor  at  the 
Defense  Systems  Management  College. 
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everyday  work,  these  tightly  connected,  complementary 
systems  help  us  manage,  think  and  make  decisions.  The 
neocortex  is  where  much  of  our  conscious  thought  occurs. 
There  short-term  memories  are  stored  and  processed.  The 
limbic  system  takes  on  our  more  primal  challenges  of  de¬ 
ciding  whether  to  fight  or  flee  in  the  presence  of  perceived 
danger.  Long-term  memories  and  experiences  also  are  stored 
here,  often  along  with  their  emotional  connections  to  pain 
or  pleasure,  past  success  or  failure.  The  two  portions  of  the 
brain  also  are  quite  different  in  their  contributions  to  our  con¬ 
scious  behaviors.  The  neocortex  engages  when  we  attempt 


favorite  decaf  mocha  soy  cappuccino  at  the  corner  coffee  shop 
without  comparing  prices,  or  evaluating  the  health  benefits  of 
soy  over  whole  milk,  emerge  from  the  limbic  system  almost 
automatically.  This  may  be  especially  true  if  your  coffee's  flavor 
is  linked  to  a  pleasant  taste  or  memory  of  that  time  you  and 
your  significant  other  visited  in  Rome. 

Likewise,  some  of  your  decisions  at  work  may  emerge  sur¬ 
prisingly  quickly.  Your  brain  may  recall  memories  and  con¬ 
nections  from  a  similar  situation  in  the  past.  If  that  past  deci¬ 
sion  got  you  into  trouble,  your  emotional  brain  may  deliver 


“A  man's  got  to  know  his  limitations." 

—Harry  Callahan  (Clint  Eastwood) 
in  "Magnum  Force,"  1973 


to  do  something  new  and  challenging.  The  thinking  portion 
of  our  brain  tends  to  be  slow,  methodical  and  logical. 

Consider,  for  example,  when  you  were  first  learning  to  drive  a 
car.  You  probably  were  overwhelmed  with  all  the  things  you 
had  to  think  about— operating  the  steering,  accelerator,  brake; 
keeping  vigilant  watch  forward  and  in  the  rearview  mirrors; 
planning  your  route,  etc. 

Our  limbic  system,  on  the  other  hand,  prompts  quicker  and 
more  automatic  actions  and  sometimes  more  visceral  "gut" 
reactions.  One  might  expect  this  from  the  portion  of  our  brain 
that  evolved  from  caveman  times  to  make  fast  life-or-death 
decisions.  In  modern  human  beings  who  are  less  likely  to  be 
lunch  for  some  saber-toothed  tiger,  the  limbic  system  has 
evolved  into  a  storehouse  for  rules  of  thumb,  or  heuristics, 
that  we  rely  on  to  make  most  of  our  routine  decisions  without 
really  thinking  about  them. 

Returning  to  the  driving  example,  it  is  likely  that  after  years 
of  practice,  you  have  had  the  experience  of  arriving  at  the  of¬ 
fice  with  few  conscious  memories  of  having  done  any  of  the 
mechanical  process  steps  during  your  drive.  You  have  moved 
thinking  about  driving  the  car  from  the  conscious  neocortex 
to  the  subconscious  limbic  system,  making  driving  virtually  a 
"no  brainer." 

Maximize  Your  Brain  Power 

Given  this  amazing  organ  and  its  two  decision-making  sys¬ 
tems,  how  can  we  best  leverage  the  way  the  brain  works  to 
optimize  the  things  we  have  it  work  on? 

Know  which  system  in  your  brain  is  making  the  decision.  Most 
of  our  routine  decisions,  including  habits  like  ordering  your 


a  strong,  fast  recommendation  that  we  recognize  as  a  "gut 
reaction."  If  the  situations  truly  are  similar,  then  going  with 
your  gut  is  probably  fine.  However,  if  something  is  subtly  dif¬ 
ferent  this  time,  it  may  be  better  to  slow  down  your  decision 
making  and  engage  your  neocortex  to  look  at  the  facts  and 
analyze  the  situation  a  bit  more.  It  will  require  a  conscious 
effort  on  your  part  to  have  your  thinking  brain  overrule  the 
limbic  system,  especially  if  a  strong  emotion  is  associated 
with  the  situation. 

If  we  have  to  guard  against  our  emotional  brain  taking  over, 
why  then  do  we  not  engage  our  neocortexes  for  every  deci¬ 
sion?  The  answer,  as  you  might  imagine,  is  that  this  strategy 
would  slow  your  life  to  a  crawl,  require  a  lot  of  time  and 
effort  to  make  even  simple  decisions,  and  likely  paralyze 
your  decision  making.  Instead,  you  should  save  your  compu¬ 
tational  brainpower  for  really  hard  problems,  new  tasks  and 
important  decisions.  It  generally  isn't  hard  to  know  which 
problems  require  real  concentration  and  thought,  but  recog¬ 
nize  that  those  decisions  will  require  more  time  and  focus. 

Manage  Your  Energy:  The  neocortex  is  not  only  slow  but  also 
an  energy  hog.  It  consumes  a  lot  of  glucose  and  oxygen  to 
accomplish  higher-order  tasks  like  planning,  assessing  and 
decision  making,  so  do  what  you  can  to  address  complex  tasks 
only  during  your  peak  brain  times.  Are  you  a  morning  person 
or  a  night  owl?  Use  these  times  and  block  your  calendar  for 
brain-draining  tasks.  Don't  try  to  multitask.  Turn  off  your  email 
and  avoid  other  distractions.  Be  aware  that  each  task  or  deci¬ 
sion  reduces  your  energy  reserve,  so  that  subsequent  efforts 
become  even  more  challenging.  For  this  reason,  schedule  15- 
to  30-minute  breaks  between  thinking  sessions  or  decision 
meetings  to  allow  your  neocortex  to  rest  and  recover.  Don't 
skip  meals.  Stay  hydrated.  It  matters. 
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Figure  I.  The  Brain 


Get  Stuff  Out  of  Your  Head:  Short-term  memory  is  very  lim¬ 
ited,  so  find  a  way  to  capture  things  like  to-do  items,  ideas,  and 
other  thoughts  on  paper,  in  your  computer  or  smart  phone. 
As  simple  as  it  sounds,  writing  down  a  comprehensive  to-do 
list  may  be  the  most  important  step  in  clearing  your  brain  of 
clutter  so  you  can  focus  on  more  important  matters.  The  book 
Getting  Things  Done  (GTD)  by  David  Allen  can  help  you  set  up 
systems  to  do  this  and  clear  you  brain  for  more  useful  tasks. 
As  Allen  says,  "Your  brain  is  for  having  ideas,  not  for  holding 
them." 

Practice  Makes  Perfect:  Transitioning  important  informa¬ 
tion  and  process  knowledge  from  short-term  memory  in  the 
neocortex  to  long-term  memory  in  the  limbic  system  through 
practice  and  reinforcement  helps  the  brain  form  time-saving 
heuristics  (or  rules  of  thumb)  and  builds  subject-matter  exper¬ 
tise.  Malcolm  Gladwell,  noted  author  and  researcher,  claims 
that  it  takes  10,000  hours  of  deliberate  practice  to  build  ex¬ 
pertise.  Experts  often  can  make  insightful  judgments  and  deci¬ 
sions  without  appearing  to  think  too  hard,  since  these  insights 
come  from  a  different  part  of  the  brain  trained  to  sort  out  com¬ 
plex  problems  by  recognizing  patterns  and  relationships  de¬ 
veloped  through  years  of  experience.  Consider  how  quickly  an 
experienced  program  manager  or  contracting  officer  can  spot 
anomalies  and  problems  in  a  Statement  of  Work  that  would 
be  beyond  an  intern's  capability.  Author  Gary  Klein  calls  this 
"seeing  the  invisible,"  and  it  is  a  skill  reserved  for  those  who 
invest  the  time  and  effort  to  become  experts  in  their  fields. 

A  Picture  Is  Worth  a  Thousand  Words:  Since  long-term 
memories  are  stored  in  the  emotional  brain,  linking  informa¬ 
tion  to  emotions  can  aid  in  this  transformation.  Many  memory 
experts  suggest  creating  vivid  (and  sometimes  outrageous) 
mental  images  linked  to  ordinary  information  in  order  to  help 
remember  them.  This  portion  of  our  brain  is  particularly  at¬ 
tuned  to  interpreting  and  remembering  pictures  and  meta¬ 
phors.  Charts  and  graphs  are  quicker  and  easier  to  grasp 


than  spreadsheets.  Diagrams  and  flowcharts  can  demystify 
complex  processes.  Mind  maps  are  more  brain-friendly  than 
outlines.  Similarly,  word  pictures  and  metaphors  are  powerful 
and  memorable.  Abraham  Lincoln  was  a  gifted  speaker  who 
frequently  used  metaphors.  He  once  described  the  continu¬ 
ous  flow  of  reinforcements  requested  by  one  of  his  Civil  War 
generals  as  "shoveling  flies  across  a  barnyard."  An  entertaining 
introduction  to  memorizing  information  by  using  vivid  images 
and  other  aids  can  be  found  in  Joshua  Foer's  book,  Moonwalk¬ 
ing  With  Einstein. 

Problem  Solving  Made  Easy:  The  limbic  system  has  hundreds 
of  millions  of  nonlinear  associations  that  are  created  and  re¬ 
formed  constantly.  Sometimes,  solutions  to  sticky  problems 
seem  to  appear  from  nowhere  in  flashes  of  intuition.  These 
"Aha!"  moments  can  occur  when  the  limbic  system  makes 
unexpected  and  subconscious  neural  associations.  This  very 
often  happens  during  idle  times  when  the  neocortex  is  not 
focused  on  the  problem  at  hand,  allowing  the  limbic  system 
room  to  work.  Some  experts  suggest  that  we  might  be  able 
to  prompt  these  amazing  insights  by  writing  down  a  specific 
question  about  a  problem  or  dilemma  and  then  posing  that 
question  to  the  brain  before  going  to  sleep  or  when  rising  in 
the  morning.  The  brain  subconsciously  looks  for  associations 
to  answer  the  question  without  necessarily  engaging  the  con¬ 
scious  brain.  When  least  expected,  and  almost  always  when 
you're  in  the  shower,  exercising,  or  on  a  break,  the  brain  de¬ 
livers  a  brilliant  flash  of  insight.  If  you're  stuck  on  a  problem, 
take  a  short  walk  and  you  may  be  rewarded  with  a  productive 
thought.  Oh,  and  keep  a  notebook  handy  to  write  it  down. 

Summary  and  Recommendations 

You  may  not  have  10,000  hours  of  experience  with  neuro¬ 
science,  but  you  do  have  some  experience  and  expertise  op¬ 
erating  your  own  brain.  With  these  tips  and  some  practice, 
you  can  have  a  reasonable  shot  at  becoming  an  expert  thinker 
and  sound  decision  maker.  Remember  to  maximize  your  brain 
power  and  decision-making  capabilities  by  managing  your 
energy  levels  and  focusing  your  efforts.  Avoid  multitasking. 
Segregate  the  more  difficult  thinking  and  deciding  tasks  to 
times  when  you  are  at  your  peak,  and  fill  the  rest  with  less 
brain-stressing  work.  Clear  your  mind  by  writing  down  all  your 
to-do  items  and  miscellaneous  notes  rather  than  trying  to  re¬ 
member  them.  Recognize  that  real  experts  make  decisions 
from  a  different  part  of  their  brains  that  have  been  honed  over 
time  to  make  deep  and  varied  connections  through  intentional 
practice,  and  get  started  building  your  own  expertise. 

Finally,  give  yourself  some  time  and  space  and  expect  those 
"Aha!"  moments  when  you  take  a  break  from  thinking  about 
your  difficult  challenges.  Know  your  limitations  and  organize 
your  work  and  life  to  maximize  your  brain's  potential.  You 
will  be  better  thinkers  and  decision  makers  when  you  follow 
these  steps.  & 


The  authors  can  be  contacted  at  roy.wood@dau.mil  and  patrick.barker@ 
dau.mil. 
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